Guidance for RNA-seq co-expression network construction and analysis: safety in numbers
Open accessAbstract
Motivation: RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined. We assessed a variety of RNA-seq expression data to determine factors affecting functional connectivity and topology in co-expression networks. Results: We examine RNA-seq co-expression data generated from 1970 RNA-seq samples using a Guilt-By-Association framework, in which genes are assessed for the tendency of co-expression to reflect shared function. Minimal experimental criteria to obtain performance on par with microarrays were >20 samples with read depth >10 M per sample. While the aggregate network constructed shows good performance (area under the receiver operator characteristic curve ∼0.71), the dependency on number of experiments used is nearly identical to that present in microarrays, suggesting thousands of samples are required to obtain ‘gold-standard’ co-expression. We find a major topological difference between RNA-seq and microarray co-expression in the form of low overlaps between hub-like genes from each network due to changes in the correlation of expression noise within each technology. Contact: jgillis@cshl.edu or sballouz@cshl.edu Supplementary information: Networks are available at: http://gillislab.labsites.cshl.edu/supplements/rna-seq-networks/ and supplementary data are available at Bioinformatics online.
AI snapshot
Written from the paper’s full text.
TL;DR
This paper benchmarks RNA-seq co-expression network construction against microarrays and finds that simple correlation-based networks, especially when aggregated across many datasets, recover known gene function comparably to microarrays. However, RNA-seq and microarray networks differ sharply in topology, largely because low-expression noise creates different hub genes.
Key findings
- Individual RNA-seq co-expression networks performed modestly, with an average AUROC of 0.585; only 0.7% of GO terms averaged above 0.8 AUROC, and only 2% ever exceeded 0.9 AUROC in any assessed network.
- Aggregating networks improved functional connectivity to about 0.71 AUROC for both RNA-seq and microarray data; roughly 20 experiments were enough for AUROC >0.6, while capturing unknown functional information may require more than 50 experiments.
- RNA-seq and microarray aggregate networks had negatively correlated node degrees, and hub-like genes were significantly dissimilar between technologies (P < 1.4e-9), even though both captured individual GO functions similarly.
- Low-expressing transcripts that failed SEQC reliability thresholds in at least 80% of experiments had high node degree in microarrays but low node degree in RNA-seq, explaining much of the topological mismatch.
Methods
- The authors assembled 50 RNA-seq experiments totaling 1970 samples from SRA, Gemma, BrainSpan, and ReCount, plus 43 microarray experiments totaling 5134 arrays from GEO.
- Expression was estimated with Cufflinks, RSEQtools, and RSEM; co-expression networks used rank-transformed correlation edge weights, with comparisons to mutual information, ARACNE, CLR, MRNET, and WGCNA thresholding.
- Functional connectivity was assessed by guilt-by-association using neighbor voting and other machine-learning classifiers, with 3-fold cross-validation and AUROC against GO, KEGG, and Reactome annotations; aggregate networks were built by summing rank-standardized edges across datasets.
Limitations
- Stated: the authors note that their use of GO as the definition of function is the most easily critiqued aspect, although KEGG and Reactome gave nearly identical results; they also state that they cannot readily assess RNA-seq in its most novel applications, such as novel or more diverse transcript assessment.
- Stated: they do not claim to identify an ideal RNA-seq co-expression method, and they note that focused uses such as tissue-specific or developmental analyses may not be higher performing in their framework.
- Evident: the study is limited to human datasets and existing annotations, ignores alternate isoforms in aggregates and performance measures, and uses median values when multiple measures exist, so isoform-level or cross-species generalization is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 17 cited sources checked · 0 supported · 0 flagged
- Findingown data
Network topology was not fully robust to the methods used, with node-degree correlations between networks ranging from -0.39 to 0.88.
“Network topology was not fully robust to the methods used (Supplementary Fig. S1A), with the correlations of node degree between the networks created ranging between q ¼ À0.39 and q ¼ 0.88.”
Quote in the paperSubstantial claim - Findingown data
Microarrays show a linear relationship between sample size and network performance (q=0.66).
“Samples assayed by microarrays show a linear relationship between sample size and network performance (Fig. 3A, q ¼ 0.66)”
Quote in the paperSubstantial claim - Findingown data
A similar but weaker trend is visible with increasing read depth.
“A similar, albeit weaker trend is visible with increasing read depth (Supplementary Fig. S2C)”
Quote in the paperSubstantial claim - Findingown data
Both RNA-seq and microarray networks converge to similar performance measures, average AUROC ~0.7.
“A notable point is that both RNA-seq and microarray networks appear to converge to similar performance measures (average AUROC $0.7).”
Quote in the paperNot a substantial claim - Findingown data
Mapping and machine learning methods affect primarily network topology and not GBA.
“We observe a particularly strong role for number of samples and aggregation (Sections 3.2 and 3.3) in GBA assessments while mapping and machine learning methods (Sections 3.1 and 3.4) affect primarily topology and not GBA.”
Quote in the paperSubstantial claim
- Findingown data
The strength of the sample-size/performance trend is consistent between microarrays and RNA-seq.
“Networks constructed using more samples were generally higher performing, and the trend is surprisingly consistent in strength between microarrays and RNA-seq (Fig. 3A and B).”
Quote in the paperSubstantial claim - Findingown data
Individual networks did not work well for different functions to any substantial extent.
“While in theory it is possible that individual networks work well for different functions, this did not occur to any substantial extent.”
Quote in the paperSubstantial claim - Findingown data
Networks constructed using more samples were generally higher performing.
“Networks constructed using more samples were generally higher performing, and the trend is surprisingly consistent in strength between microarrays and RNA-seq (Fig. 3A and B).”
Quote in the paperSubstantial claim - Findingown data
For any given experiment, the fraction of its GO terms that outperform the aggregate is very small.
“Similarly, for any given experiment, the fraction of its GO terms which outperform the aggregate is very small, reaching a maximum of 17% only in very large experiments and averaging only 5% of GO terms (again, those with low performance, Fig. 5B).”
Quote in the paperSubstantial claim - Findingown data
The top 23 performing networks aggregated had the highest performance (average AUROC=0.71), only just above the random sampled range.
“The top group had the highest performance (average AUROC ¼ 0.71), only just above the range of the random sampled”
Quote in the paperSubstantial claim - Findingown data
The collagen complex (GO:0005581) gives moderate performance in underlying datasets (mean AUROCs ~0.7).
“The 'collagen' complex (GO: 0005581) gives moderate performance in the underlying datasets (mean AUROCs $0.7), but benefits strongly from aggregation (AUROC >0.9).”
Quote in the paperNot a substantial claim - Findingown data
Alternative MI inference methods (ARACNE additive, ARACNE multiplicative, CLR, MRNET) performed considerably worse than correlation coefficient inference methods, with average AUROCs 0.52, 0.53, 0.55, and 0.55 respectively.
“When we also tried the alternative MI inference methods such as ARACNE, CLR and MRNET, the networks also performed considerably worse than the correlation coefficient inference methods (e.g.average AUROCs for ARACNE-additive ¼ 0.52, ARACNE-multiplicative ¼ 0.53, CLR ¼ 0.55 and MRNET ¼ 0.55).”
Quote in the paperSubstantial claim - Findingown data
RNA-seq co-expression networks exhibit more variability in performance as a function of sample count than microarrays (q=0.29, P=0.04).
“however RNA-seq co-expression networks exhibit more variability in performance as a function of sample count than in microarrays (Fig. 3B, q ¼ 0.29, P $ 0.04), possibly due to variation in read depth.”
Quote in the paperSubstantial claim - Findingown data
A majority (56%) of GO terms never outperform the aggregate in any experiment.
“However, a majority (56%, Fig. 5A) of GO terms never outperform the aggregate in any experiment.”
Quote in the paperSubstantial claim - Findingreasoning
There is a major topological difference between RNA-seq and microarray co-expression in the form of low overlaps between hub-like genes from each network.
“We find a major topological difference between RNA-seq and microarray co-expression in the form of low overlaps between hub-like genes from each network”
Quote in the paperSubstantial claim - Findingown data
In the aggregate, an average connection exceeds the connection strength in just 35 of the 50 datasets.
“On average, a given connection in the aggregate is only more tightly connected than in 35 out of the 50 underlying datasets”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Findingown data
Only 46 out of 2953 GO groups perform better on average in individual datasets than the aggregate.
“Indeed, only 46 (out of 2953) GO groups perform better on average in the individual datasets than the aggregate, and these functions are ones with very poor performance in both the aggregate and underlying data (average AUROC <0.52 in the individual datasets, Supplementary Table S3).”
Quote in the paperSubstantial claim - Findingown data
There is slower convergence to a fixed node degree under aggregation.
“This is further suggested by a slower convergence to a fixed node degree under aggregation (Supplementary Fig. S5A).”
Quote in the paperNot a substantial claim - Findingown data
Protein complexes were enriched when ranking GO terms by the degree to which they benefited from aggregation (Wilcoxon p~0.01).
“and complexes were enriched when ranking terms by the degree to which they benefited from aggregation (Wilcoxon test, p $ 0.01).”
Quote in the paperSubstantial claim - Findingown data
The expression profiles and networks from the three expression estimation methods were modestly correlated.
“We see that these methods generated networks with similar performances (Supplementary Fig. S1B), even though the expression profiles and networks were mod- estly correlated (Supplementary Fig. S1C).”
Quote in the paperNot a substantial claim - Findingown data
Aggregates of the initial 10 experiments all had similar functional connectivity as determined by GO, with average AUROCs at 0.67.
“The aggregates of only those 10 experiments also all had similar functional connectivity as determined by GO, with average AUROCs at 0.67.”
Quote in the paperNot a substantial claim - Findingown data
Aggregates of the initial 10 subset experiments had high correlations between node degrees across expression estimation methods (q=0.54-0.84).
“Even though the individual experiments had weaker correlations, the correlations between the node degrees of the aggregates of each of the methods were generally high (q ¼ 0.54-0.84).”
Quote in the paperSubstantial claim - Findingown data
The GO groups that perform better in individual datasets than the aggregate have very poor performance in both aggregate and underlying data (average AUROC <0.52 in individual datasets).
“Indeed, only 46 (out of 2953) GO groups perform better on average in the individual datasets than the aggregate, and these functions are ones with very poor performance in both the aggregate and underlying data (average AUROC <0.52 in the individual datasets, Supplementary Table S3).”
Quote in the paperSubstantial claim - Findingown data
Aggregation across experiments improves performance well beyond that attained by even the largest individual experiments with far fewer samples.
“Aggregation across experiments improves performance well beyond that attained by even the largest individual experiments with far fewer samples.”
Quote in the paperSubstantial claim - Findingown data
Topologically important genes vary readily.
“We explain the discrepancy by finding topologically important genes vary readily (Section 3.5), but primarily due to noise in the data (Section 3.6).”
Quote in the paperSubstantial claim - Findingown data
The collagen complex benefits strongly from aggregation (AUROC >0.9).
“The 'collagen' complex (GO: 0005581) gives moderate performance in the underlying datasets (mean AUROCs $0.7), but benefits strongly from aggregation (AUROC >0.9).”
Quote in the paperNot a substantial claim - Findingown data
The dependency on number of experiments used is nearly identical to that present in microarrays.
“the dependency on number of experiments used is nearly identical to that present in microarrays”
Quote in the paperSubstantial claim - Findingown data
WGCNA soft thresholding lowers network performance, with average AUROC 0.55.
“Also, the WGCNA soft thresholding acts in a similar fashion, lowering performance (average AUROC ¼ 0.55).”
Quote in the paperSubstantial claim - Findingown data
The average AUROC performance of random experimental sets of 23 networks ranged between 0.69 and 0.71 (mean 0.70, SD~0.01).
“The average AUROC performance of the random experimental sets ranged between 0.69 and 0.71 (mean average AUROC ¼ 0.70, SD $0.01).”
Quote in the paperSubstantial claim - Findingown data
Cufflinks, RSEQtools, and RSEM generated networks with similar functional performance.
“We see that these methods generated networks with similar performances (Supplementary Fig. S1B), even though the expression profiles and networks were mod- estly correlated (Supplementary Fig. S1C).”
Quote in the paperSubstantial claim - Findingown data
In the collagen complex, co-expression performance is broadly distributed both between gene pairs and across datasets.
“Performance in the collagen complex is very broadly distributed both between gene pairs and across different datasets.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Findingown data
Only 0.7% of GO terms had average performance above AUROC 0.8 across all networks.
“While some GO groups exhibited good performance across all networks, they were a relative minority, with only 0.7% of GO terms having average performances above 0.8 (AUROC, Fig. 2A).”
Quote in the paperSubstantial claim - Findingown data
The aggregate network constructed shows good performance (AUROC 0.71).
“While the aggregate network constructed shows good performance (area under the receiver operator characteristic curve $0.71)”
Quote in the paperSubstantial claim - Findingown data
Across individual experiments, co-expression networks exhibit only modest performance, tightly clustered around average AUROC 0.585.
“Across our corpus of individual experiments, co-expression networks constructed exhibit only modest performance, being fairly tightly clustered around an average AUROC of 0.585.”
Quote in the paperSubstantial claim - Findingown data
Minimal experimental criteria to obtain performance on par with microarrays were >20 samples with read depth >10 M per sample.
“Minimal experimental criteria to obtain performance on par with microarrays were >20 samples with read depth >10 M per sample.”
Quote in the paperSubstantial claim - Findingown data
For each connection, the diffuse gain between gene pairs relies on a substantial portion of the data.
“This diffuse improvement between gene-pairs depends on a large fraction of the data for each connection.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Findingown data
Controlling for read depth by sampling within large experiments shows a very similar sample-size performance trend, shifted upward by about 0.02 AUROC, for both microarray and RNA-seq.
“Controlling for this by sampling from within large experiments to calculate the dependency on sample size shows a very similar performance trend, albeit shifted upward (AUROC gain of $0.02) for both microarray and RNA-seq.”
Quote in the paperSubstantial claim - Findingown data
Correlations between GO AUROCs for the aggregates of the initial 10 experiments were high, ranging from 0.86 to 0.88.
“The correlations between the GO AUROCs were also high, ranging between q ¼ 0.86 and 0.88 (Supplementary Table S5).”
Quote in the paperNot a substantial claim - Findingown data
Repeating functional connectivity experiments using KEGG and Reactome as references gave essentially identical results.
“We repeated the same functional connectivity experiments using KEGG and Reactome as references, and had essentially identical results (Supplementary Figs.S3 and S4).”
Quote in the paperNot a substantial claim - Findingown data
The number of samples and aggregation have a strong role in GBA assessments.
“We observe a particularly strong role for number of samples and aggregation (Sections 3.2 and 3.3) in GBA assessments while mapping and machine learning methods (Sections 3.1 and 3.4) affect primarily topology and not GBA.”
Quote in the paperSubstantial claim - Findingown data
The bottom 23 performing networks aggregated had much lower performance (average AUROC=0.66).
“while the bottom 23 group had a much lower performance (average AUROC ¼ 0.66).”
Quote in the paperSubstantial claim - Findingown data
Few high-performing GO terms existed outside this minority.
“Moreover, there were few high performing GO terms outside of this minority.”
Quote in the paperSubstantial claim - Findingown data
Cross-platform node-degree correlations were much lower (RNA-seq to microarray aggregate mean q = -0.02; microarray to RNA-seq aggregate mean q = 0.06) than same-platform correlations (RNA-seq to RNA-seq mean q = 0.38; microarray to microarray mean q = 0.26).
“Overall, cross platform correlations are much lower (RNA-seq networks to microarray aggregate average q ¼ À0.02, microarray networks to RNA-seq aggregate average q ¼ 0.06) than the correlations between the aggregate and the individual networks of the same platform (RNA-seq networks to RNA-seq aggregate average q ¼ 0.38, microarray networks to microarray aggregate average q ¼ 0.26).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
The sets of hub-like genes (top 3%, about 500 genes) differed significantly between the two networks (P < 1.4e-9).
“top 3%, $500 genes) were significantly dissimilar in the two networks (P < 1.4eÀ9).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
Transcripts failing the SEQC threshold in at least 80% of experiments had very different topology in the two platforms: high node degree in microarray and low in RNA-seq.
“This set of transcripts exhibits starkly different topological properties in the RNA-seq co-expression and microarray data, being very high node degree within microarray and very low in RNA-seq (Fig. 6B).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
Using KEGG and Reactome as references gave nearly identical results.
“Since our results were virtually identical using KEGG and Reactome as references”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Findingown data
Node degree correlation between the RNA-seq and microarray aggregates was negative (q < 0) for every methodology.
“the correlation between the node degree between the RNA-seq aggregate and the microarray aggregate was negative (q < 0) across all methodologies”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
Individual RNA-seq networks show sample-size dependence similar to that of microarray data.
“individual RNA-seq networks exhibit a sample size dependency very similar to microarray data”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
Applying the same SEQC-based criterion to microarray did not produce a node-degree difference between microarray and RNA-seq co-expression.
“If we use the same criterion to define a set of transcripts (failing SEQC guidelines but in microarray), we see no such difference in node degree between microarray and RNA-seq co-expression (Fig. 6C).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
The co-occurrence network had very poor functional connectivity (mean AUROC = 0.56).
“the functional connectivity of the co-occurrence network is very poor (average AUROC ¼ 0.56, Supplementary Fig. S2H).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
The two networks captured individual GO functions similarly (P = 0.84).
“Even so, both networks capture individual GO functions in a similar way (P ¼ 0.84, Fig. 4B and inset).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Findingown data
No evidence was found that more focused uses would perform better in any way.
“While our analysis does not find evidence to support the view that such uses will be higher performing in any way”
Quote in the paperNot a substantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Findingown data
Node degrees of the co-occurrence null network correlated with the true co-expression network at q = 0.45.
“The correlation between node degrees of this network and the true co-expression network (q ¼ 0.45, Supplementary Fig. S5D)”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
The results are not strongly method independent.
“implying that the results are not strongly method independent.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Interpretationreasoning
Thousands of samples are required to obtain 'gold-standard' co-expression.
“suggesting thousands of samples are required to obtain 'gold-standard' co-expression.”
Quote in the paperSubstantial claim - Premisecitation
Without careful experimental design, it is difficult to separate gene expression variability caused by technical artifacts from biological signal.
“In the absence of careful experimental design, distinguishing gene expression variability due to technical artifacts and biological signal is challenging (Auer and Doerge, 2010).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Auer and Doerge, 2010 source
Not checked in the free run.
- Premisenone given
The presumed novelty of RNA-seq co-expression networks has not been fully assessed.
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
Quote in the paperNot a substantial claim0 of 2 sources checked - Premisecitation
Few conclusions have been drawn about the utility of RNA-seq co-expression networks.
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Even gold-standard expression datasets perform relatively poorly under GBA in benchmarking exercises.
“Even 'gold-standard' expression datasets perform relatively poorly under GBA when assessed in benchmarking exercises (e.g. the Mousefunc competition for genome-wide gene function prediction- Pena-Castillo et al., 2008).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Pena-Castillo et al., 2008 source
Not checked in the free run.
- Interpretationown data
The variation in topologically important genes is primarily due to noise in the data.
“We explain the discrepancy by finding topologically important genes vary readily (Section 3.5), but primarily due to noise in the data (Section 3.6).”
Quote in the paperSubstantial claim - Premisecitation
Genes with similar functional properties tend to interact or show similar profiles in network data such as co-expression.
“GBA states that genes with similar functional properties will tend to interact or exhibit similar profiles in network data, such as co-expression.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Oliver, 2000 source
Not checked in the free run.
- Interpretationown data
To obtain even modest RNA-seq co-expression performance (AUROC >0.55), sample sizes >20 and read depth per sample close to 100 M reads are required.
“and suggests using RNA-seq co-expression to obtain even modest performance (AUROC >0.55) requires sample sizes >20 and read depth per sam- ple close to 100 M reads.”
Quote in the paperSubstantial claim - Interpretationown data
About 20 separate experiments with moderate depth and sample count >10 are sufficient to obtain a high-quality network (AUROC >0.6) as captured by GO.
“While $20 separate experiments (with moderate depth and sample count >10) is sufficient to obtain a high-quality network (AUROC >0.6) as captured by GO”
Quote in the paperSubstantial claim - Interpretationown data
To comprehensively capture unknown functional information may require more than 50 separate experiments.
“to comprehensively capture unknown functional information may require >50 separate experiments.”
Quote in the paperSubstantial claim- Rests on an assumption: The aggregate network's performance on GO terms is a valid proxy for capturing unknown functional information.
- Interpretationown data
Aggregate performance is spread across numerous pairwise relationships of moderate significance.
“This suggests, once again, that the very high performance in aggregate is distributed throughout many moderately significant pairwise relationships.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Study design and conductnone given
Changes in the distribution of actual correlations caused by sample size dependencies do not affect the study's results.
“Changes in the distribution of actual correlations (induced by sample size dependencies) have no impact on our results; only the relative strength of gene-gene correlations within an experiment alters network topology.”
Quote in the paperNot a substantial claim - Interpretationnone given
The low overlap between hub-like genes from RNA-seq and microarray networks is due to changes in the correlation of expression noise within each technology.
“due to changes in the correlation of expression noise within each technology.”
Quote in the paperNot a substantial claim - Study design and conductnone given
Only the relative strength of gene-gene correlations within an experiment alters network topology.
“Changes in the distribution of actual correlations (induced by sample size dependencies) have no impact on our results; only the relative strength of gene-gene correlations within an experiment alters network topology.”
Quote in the paperSubstantial claim - Premisenone given
Very few analyses of co-expression networks from RNA-seq data have been performed.
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Meta-analysis across many datasets often improves performance dramatically.
“However, meta-analysis across many datasets often improves performance dramatically (Gillis and Pavlidis, 2011a;Lee et al., 2004;Wren, 2009).”
Quote in the paperSubstantial claim0 of 3 sources checked - Interpretationreasoning
Because co-expression often aggregates diverse data, many issues with RNA-seq as a pure transcriptomic measure may not matter downstream in co-expression.
“Co-expression has frequently relied on the aggregation of diverse data and so many of the problems that affect RNA-seq as a pure measure of transcriptomic activity may not have downstream importance in co-expression use.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Interpretationown data
The topological shuffling between RNA-seq and microarray is interpreted not as a robust difference but as measurements being easily swayed by weak biases such as gene size.
“Our interpretation of the topological shuffling we see between RNA-seq and microarray is not that they are robustly different (even though the significance is high), but rather that the measurements are easily influenced by weak biases (e.g.gene size).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
The presence of a gene pair strongly influences higher node-degree correlation.
“implies that there is a strong influence of the presence of a gene pair contributing to a higher node degree correlation.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
The noisiness of co-expression data does not primarily stem from technical issues.
“suggesting that technical issues are not at the heart of the 'noisiness' of co-expression data.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationnone given
RNA-seq topology is most stable where artifacts probably dominate, such as within pipelines or co-occurrence in meta-analysis.
“Moreover, RNA-seq is most robust in its topology where artifacts likely dominate, such as in within given pipelines or cooccurrence in meta-analysis.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
Genes that were highly connected in one network often lost that connectivity in the other.
“implying that many highly connected genes in one network were now no longer well connected in the second.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
Topological differences between microarray and RNA-seq are driven by technical variation.
“Altogether, this strongly suggests that technical variation drives the topological differences between microarray and RNA-seq.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Premisecitation
SEQC defines conditions under which RNA-seq transcript expression estimates are considered unreliable, with the bottom third of transcripts by expression as a key threshold.
“SEQC provides criteria under which transcript expression estimates are not expected to be reliable in RNA-seq pipelines (with moderately low expression-the bottom one-third of transcripts-being a major threshold).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Li et al., 2014
Not checked in the free run.
- cites SEQC/MAQC-III Consortium, 2014 source
Not checked in the free run.
- Interpretationown data
These transcripts supply many hub-like genes in microarray, which fully explains the lack of overlap.
“Because these transcripts contribute many hub-like genes to the microarray data, this effect is fully sufficient to account for the lack of overlap we see.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
The findings are probably robust across most reference data choices.
“we suggest our results are likely robust to most choice of reference data.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Interpretationown data
The topological reordering between microarray and RNA-seq is due to noisy expression.
“Altogether, this strongly suggests that the topological reordering we see between microarray and RNA-seq reflects noisy expression”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- Interpretationown data
RNA-seq detects low, unreliable expression more sensitively.
“RNA-seq is both more sensitive at detecting when this occurs (when expression is too low to be reliable)”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
- May combine two claims. 33
- Classified as probably not a substantial claim. 6
May combine two claims.
Findingown dataTaking all protein complex subgroups in GO, their average performance in the aggregate was AUROC ~0.77, significantly greater than the remaining terms (Wilcoxon p~5.423e-7).
“Taking all the protein complex subgroups in GO in our analyses, their performance on average in the aggregate was AUROC $0.77, significantly greater than the remaining terms (Wilcoxon test, p $ 5.423eÀ7)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe maximum fraction of GO terms outperforming the aggregate in an experiment is 17%, only in very large experiments, and the average is 5%.
“Similarly, for any given experiment, the fraction of its GO terms which outperform the aggregate is very small, reaching a maximum of 17% only in very large experiments and averaging only 5% of GO terms (again, those with low performance, Fig. 5B).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMutual information networks perform below correlation coefficient networks, closer to networks including negative correlation, with average AUROC 0.56.
“The MI network performs below the correlation coefficients, and closer to the networks generated including the negative correlation (average AUROC ¼ 0.56).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAggregating across many networks improves performance and removes noise.
“## 3.3 Aggregating across many networks improves performance and removes noise”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe correlation between GO AUROCs in RNA-seq and microarray aggregates is 0.84 (P<1e-16), indicating GO groups that perform well in one network perform equally well in the other.
“Inset shows the correlation between the two GO AUROCs distributions plotted (q ¼ 0.84, P < 1eÀ16), showing that GO groups that perform well in one, perform equally well in the other network”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAcross machine learning methods, average AUROC differed modestly: neighbor voting, logistic regression, and stochastic gradient descent were best (mean AUROC 0.69), while random forest was worst (mean AUROC 0.63).
“The average AUROC varied slightly between methods (Supplementary Table S4), with the neighbor voting, logistic regres- sion and stochastic gradient descent giving the highest performance (average AUROC ¼ 0.69), and the random forest performing the worst of the six methods (average AUROC ¼ 0.63).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingown dataThe six methods showed good performance on GO groups similar to those of neighbor voting, with correlations between q = 0.87 and 0.96 (P < 2.2e-16).
“However, all the methods perform well on similar GO groups as neighbor voting (performances correlated at q ¼ 0.87-0.96, P < 2.2eÀ16)”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingown dataIn the collagen complex aggregate, 47% of gene pairs exhibit top 10% co-expression ranking, while less than 2% exhibit top 1% ranking.
“In the aggregate, almost half (47%) of the gene pairs exhibit high ranking (top 10%) co-expression, while a proportionately smaller fraction ($2%) exhibit very highly ranked (top 1%) co-expression.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingown dataRemoving connections ranked in the top 1% did not significantly reduce performance, whereas removing connections in the top 10% had a large effect (AUROC ~0.6).
“we verified by removing the connections ranked in the top 1% and seeing no significant drop in performance whereas removal of the connections in the top 10% had a large effect (AUROC $0.6).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingnone givenFor genome-wide functional inference in co-expression, adding depth beyond 100 million aligned reads per sample and even thousands of samples still adds value.
“In co-expression analysis, increasing depth past 100 M aligned reads per sample and even thousands of samples continue to add value if our focus is on genome-wide functional inference for a given dataset.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Findingown dataRNA-seq and microarray co-expression networks are alike functionally but differ greatly in overall topology.
“RNA-seq coexpression and microarray co-expression are very similar in their functional properties but very dissimilar in their overall topology (Fig. 6A).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Findingnone givenRank-and-summate aggregation over many datasets yields a more robust network that captures known biology effectively.
“a simple rank-andsummate aggregation across multiple datasets generates a more robust network, capturing known biology with high efficacy.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingown dataWith many samples, 1 million reads per sample suffices to build a network slightly different from random, and adding such networks does not harm aggregate performance.
“Our study suggests that 1 M reads per sample (with many samples) is sufficient to build a network that is slightly divergent from random and adding such networks has no negative impact on the performance of the aggregate network.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Findingown dataBasic approaches, including Spearman correlation for connectivity and neighbor voting for functional inference, ranked among the best.
“simple, basic approaches were among the highest performing, from measuring network connectivity (Spearman correlation) to functional inference (neighbor voting).”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Findingown dataFor BrainSpan, microarray and RNA-seq had similar mean GO AUROCs (0.64 and 0.65), yet their node-degree correlation was q = -0.66, indicating stark topological difference.
“Here, again, performances are similar in microarray and RNA-seq (mean GO AUROCs of 0.64 and 0.65, respectively), but topology is starkly different with a correlation of q ¼ À0.66 between the node degrees of the two networks constructed.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataSimple correlation was the computationally simplest and most transparent method for calculating co-expression and also the highest performing.
“In summary, the computationally simplest and most transparent method for calculating co-expression-simple correlation-was also the highest performing.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationRNA-seq carries technical and biological biases, such as dependence on library preparation, sequencing depth, gene length, GC content, transcript length, and laboratory-specific practices.
“These include dependencies on library preparation, sequencing depth, gene lengths, GC content and transcript lengths, as well as other laboratory-specific practices (Hitzemann et al., 2013).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hitzemann et al., 2013 source
Not checked in the free run.
May combine two claims.
Interpretationown dataGroups benefiting from aggregation are characterized by moderate performance before aggregation and by being protein complexes.
“It is characteristic of groups benefiting from aggregation in exhibiting moderate performance even before aggregation and, interestingly, in being a protein complex.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataAggregation is most effective for functions whose pairwise relationships are distributed, such as protein complexes rather than dynamic pathways, and weak aggregate effects can sum to a strong total signal.
“Altogether, our examination of the collagen complex supports the view that aggregation will work best in functions where pairwise relationships are expected to be distributed (e.g. protein complexes rather than dynamic pathways) and weak aggregate effects can summate to provide a strong total signal.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
Classified as probably not a substantial claim. May combine two claims.
PremisecitationNode degree is central to network topology and is a major factor in numerous other network properties.
“Node degree is central to network topology and a major factor in numerous other properties (Gillis and Pavlidis, 2011b;Newman, 2003).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Gillis and Pavlidis, 2011b
Not checked in the free run.
- cites Newman, 2003 source
Not checked in the free run.
May combine two claims.
Interpretationown dataEven the weakest networks are useful in aggregate, as the bottom aggregate still outperforms almost all individual networks (equal to the best performing).
“Even though this is lower than the random sets of experiments, it still outperforms almost all of the individual networks (equal to the best performing), indicating that even the weakest networks are useful in aggregate (Supplementary Fig. S5B).”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningMoving from AUROC 0.55-0.6 requires either thousands of samples or millions of more sequenced reads, which is likely not feasible in a single experiment.
“The apparent performance dependencies suggest that to move from an AUROC ¼ 0.55-0.6 requires either thousands of samples, or millions of more sequenced reads, which is likely not feasible in a single experiment.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe aggregate network was resampled across datasets to ensure it exhibited robust properties.
“We also resampled across datasets to ensure the aggregated network exhibited robust properties.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
InterpretationreasoningRNA-seq has unique advantages but is not a universal technical solution.
“In summary, RNA-seq offers unique virtues but is not a technical panacea.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningDifferential-expression recommendations cannot be applied naively to co-expression, since sample numbers are already high and reliable correlation measurement may need even more.
“Users of co-expression data cannot naively apply those recommendations because the minimum number of samples is already relatively high, and robustly measuring correlations may require an even greater increase.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataThe negative microarray-RNA-seq correlation recurs in individual networks, not only aggregates, pointing to technological biases.
“This indicates that the negative correlation between microarrays and RNA-seq data is a recurring feature of the individual networks, and not just the aggregates, and suggests a feature due to technological biases (in conjunction with network construction methods, Supplementary Fig. S5C).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataThis difference implies that although inference methods smooth out topological noise, the networks themselves show it strongly and it can change with methodology.
“This dissimilarity suggests again that while inference methods average away topological noise, the actual networks exhibit it to a substantial degree and this could be easily altered by varying methodology in other ways.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThis absence of evidence may stem from using GO as the reference even for narrowly defined functions.
“this may reflect our reliance on GO as a reference, even for narrowly defined functions.”
Quote in the paperNot a substantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Interpretationnone givenWithout careful filtering, unsupervised interpretation of RNA-seq connectivity is probably not robust to methodological choices, whereas supervised or semi-supervised methods seem to recover known information safely.
“Unsupervised interpretation of RNA-seq data connectivity is unlikely to be robust to underlying methodological choices without careful filtering, while methods that extract signals using training data (supervised or semi-supervised methods) appear to safely recover known information.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
InterpretationreasoningBecause node-degree variation has little effect on semisupervised learning such as GBA, many methods can detect the absence of learnable features in noisy expression even without proper filtering.
“Further, the low impact of variation in node degree on semisupervised learning (e.g. through GBA) suggests that even without the appropriate filtering, many methods adequately identify the absence of learnable features in noisy expression.”
Quote in the paperNot a substantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataChoice of machine learning algorithm barely affects results, implying that the features used for prediction, connectivity, and network assessment are adequate.
“the machine learning algorithm selected has little influence on the results, which also implies that the features being selected to make predictions, determine connectivity and assess the networks are adequate for the task at hand.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
May combine two claims.
Interpretationown dataAggregate versus individual network performance differs so greatly that no minimum depth or sample count can be recommended as adequate; none tested here approached the aggregate's apparent performance.
“So profound is the performance difference between aggregate and individual networks that it is difficult to recommend any minimum depth or sample count which is adequate; certainly none in our case come close to producing the same apparent performance.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataThe reordering reflects low-expressing genes tending to correlate in microarray, while RNA-seq identifies untrustworthy low-expression genes more specifically.
“Our analysis suggests that the reordering reflects the tendency for low-expressing genes to be correlated in microarray data, but that RNA-seq allows us to identify which genes are 'too' low to be trusted with much greater specificity.”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
May combine two claims.
Interpretationown dataRNA-seq avoids creating spurious correlations among noisy low-expression values, resulting in low node degree.
“and in not generating spurious correlation among these noisy values (low node degree).”
Quote in the paperSubstantial claim- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationreasoningCo-expression studies will continue to need large sample numbers and a trustworthy reference network for control and comparison.
“Co-expression analyses will still require many samples and a reliable quality reference network as a control and comparative measure.”
Quote in the paperNot a substantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
Classified as probably not a substantial claim.
InterpretationreasoningReliance on prior knowledge (GO) or data (microarray) for reference prevents easy assessment of RNA-seq in its most novel applications, such as novel or diverse transcript analysis.
“Because of our reliance on pre-existing knowledge (GO) or data (microarray) to provide reference knowledge, we cannot readily assess RNA-seq where its application is likely to prove most novel and exciting, e.g.novel or more diverse transcript assessment.”
Quote in the paperSubstantial claim- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
The paper relies on this without support.
RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined.
“RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined.”
The paper relies on this without support.
The GO, KEGG, and Reactome annotations used as labels accurately reflect gene function and are sufficiently complete for evaluating functional connectivity.
“By using known annotations as labels for the genes (here we used GO, KEGG and Reactome labeled gene sets), one can determine how well the network recapitulates this information”
The paper relies on this without support.
The 10 RNA-seq experiments selected for method comparisons are representative of the broader corpus.
“We used a sample of 10 representative experiments, comparing the estimation expressions of Cufflinks, RSEQtools and RSEM and constructed co-expression networks using our default procedures”
The paper relies on this without support.
The aggregate network's performance on GO terms is a valid proxy for capturing unknown functional information.
“to comprehensively capture unknown functional information may require >50 separate experiments.”
The paper relies on this without support.
The RNA-seq and microarray datasets used are comparable enough that observed differences in network topology and sample-size dependency reflect technology rather than dataset composition.
“In a similar fashion, we generated an aggregate of the microarrays on a random sample of 30 datasets using 3320 individual microarrays.”
The paper relies on this without support.
GO slim's 109 terms are broadly representative of the ontology overall.
“For our machine learning analysis, we used GO slim, a version of GO that filters analyses to 109 GO terms, broadly representative of the ontology overall.”
The paper relies on this without support.
The GO reference data adequately represents biological function for evaluating co-expression network performance.
“our use of the GO as a way of defining 'function' is perhaps the most easily critiqued, albeit hard to avoid and common.”
The paper relies on this without support.
The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
“We used the 30 networks from the microarray aggregate, and the 50 networks of the RNA-seq aggregates”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Differential co-expression methods are sensitive to noise.
“While differential co-expression methods are sensitive to noise [21], they are becoming more effective with the increase in RNA-seq data quantity and quality.”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
Section 3.6 links noisy expression to changes in co-expression network topology, and the Discussion says network measurements can be influenced by weak biases. These passages support noise sensitivity in co-expression analysis generally, but the paper does not specifically assess differential co-expression methods. (LLM, conf=0.84)
Experimental — accuracy not yet measuredCo-expression networks constructed using a 10 million reads per sample cut-off have been suggested to have a similar quality to microarray-based co-expression networks if constructed from the same number of samples, but decrease in quality with fewer reads.
“Co-expression networks constructed using this cut-off have been suggested to have a similar quality to microarray-based co-expression networks if constructed from the same number of samples [21], but decreasing in quality with fewer reads.”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredA higher total read depth for RNA-seq samples increases the accuracy of the expression measurements, especially for genes with low expression.
“A higher total read depth for RNA-seq samples increases the accuracy of the expression measurements, especially for genes with low expression [21, 59].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
Section 3.2 reports that greater read depth is associated with better RNA-seq co-expression network performance, and Section 3.6 says expression estimates for low-expression transcripts may be unreliable. The paper does not establish that higher read depth increases the accuracy of expression measurements, or that any such improvement is especially large for low-expression genes. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIncreased sample sizes produce networks with a higher functional connectivity.
“To create co-expression networks from RNA-seq data, a 20-sample minimum has been suggested [21, 54], and increased sample sizes produce networks with a higher functional connectivity [21, 59].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
A 20-sample minimum has been suggested for creating co-expression networks from RNA-seq data.
“To create co-expression networks from RNA-seq data, a 20-sample minimum has been suggested [21, 54], and increased sample sizes produce networks with a higher functional connectivity [21, 59].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
Higher quality data tend to result in more accurate co-expression networks.
“Not surprisingly, higher quality data tend to result in more accurate co-expression networks [21, 59].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [21]
At currently available sample sizes, networks are most reproducible at module level, then hub gene level, and least at precise gene connectivity rankings or module memberships.
“At currently available sample sizes, networks are most reproducible at a module level35,38,39, then at the hub gene level41,223 and, last, at the level of precise gene connectivity rankings or precise module memberships of genes40,49.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [40]
To obtain module-level reproducibility, 20 independent samples are usually sufficient, but systematic and accurate reconstruction of specific edges may require hundreds of samples.
“To obtain module-level reproducibility, 20 independent samples are usually sufficient40, but systematic and accurate reconstruction of specific edges, particularly for systematic regulatory relationship discovery, may require hundreds of samples49.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [40]
Many guidelines for DGE and network analysis are based on theoretical models and empirical assessments, but most use data lacking the spatial, temporal, or disease-relevant complexities of CNS or post-mortem tissue.
“For DGE and network analysis, there are many studies with guidelines that are based on theoretical models and empirical assessments40,173–176, but most studies use data from experiments that do not have the spatial, temporal or disease-relevant complexities that occur in studies of the CNS or post-mortem tissue.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [40]
Systematic consistency at the more granular hub and module membership levels may require hundreds of samples.
“To obtain module-level reproducibility, 20 independent samples are typically considered adequate; however, systematic consistency at the more granular hub and module membership levels may require hundreds of samples [61, 93, 94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [93]
To obtain module-level reproducibility, 20 independent samples are typically considered adequate.
“To obtain module-level reproducibility, 20 independent samples are typically considered adequate; however, systematic consistency at the more granular hub and module membership levels may require hundreds of samples [61, 93, 94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [93]
Network analyses of AD brain tissue demonstrate the most reproducibility at the module level, followed by the hub protein level, and then at the level of precise protein connectivity rankings and module protein membership.
“Network analyses of AD brain tissue demonstrate the most reproducibility at the module level, followed by the hub protein level, and then finally at the level of precise protein connectivity rankings and module protein membership [61, 89–94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [93]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Conflicting minimum read-depth guidance
MajorThe abstract gives more than 10 million reads per sample as a minimum for performance comparable to microarrays. The results section says that even modest performance requires close to 100 million reads per sample, a substantially higher threshold.
“Minimal experimental criteria to obtain performance on par with microarrays were >20 samples with read depth >10 M per sample.”
“suggests using RNA-seq co-expression to obtain even modest performance (AUROC >0.55) requires sample sizes >20 and read depth per sam- ple close to 100 M reads.”
Machine-learning method dependence is described in opposite ways
MinorThe results say the findings are not strongly independent of method, whereas the discussion says the selected algorithm has little influence on them.
“implying that the results are not strongly method independent.”
“the machine learning algorithm selected has little influence on the results”
Figure 4 correlation is labeled as a P-value in the text
MinorThe text assigns P = 0.84 to the similarity shown in Figure 4B, but the figure caption identifies 0.84 as the correlation and gives its P-value as less than 1e−16.
“Even so, both networks capture individual GO functions in a similar way (P ¼ 0.84, Fig. 4B and inset).”
“Inset shows the correlation between the two GO AUROCs distributions plotted (q ¼ 0.84, P < 1eÀ16)”
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Fast gapped-read alignment with Bowtie 2
Langmead, B., & Salzberg, S. L. (2012) · Nature Methods · doi:10.1038/nmeth.1923 · Cited by 56,400 · see citing works
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206 · see citing works
KEGG: Kyoto Encyclopedia of Genes and Genomes
Kanehisa, M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.27 · Cited by 37,077 · see citing works
KEGG: Kyoto Encyclopedia of Genes and Genomes
Ogata, H., Goto, S., Sato, K., Fujibuchi, W., Bono, H., & Kanehisa, M. (1999) · Nucleic Acids Research · doi:10.1093/nar/27.1.29 · Cited by 31,232 · see citing works
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · see citing works
The Structure and Function of Complex Networks
Newman, M. E. J. (2003) · SIAM Review · doi:10.1137/s003614450342480 · Cited by 18,719 · see citing works
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556 · see citing works
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696 · see citing works
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 12,617 · see citing works
Smyth, G. K. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1027 · Cited by 11,829 · see citing works
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,614 · see citing works
LIBLINEAR: A Library for Large Linear Classification
Fan, R.-E., Chang, K., Hsieh, C., Xiang-rui, W., & Lin, C. (2008) · doi:10.5555/1390681.1442794 · Cited by 6,680 · see citing works
A General Framework for Weighted Gene Co-Expression Network Analysis
Zhang, B., & Horvath, S. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1128 · Cited by 5,684 · see citing works
GENCODE: The reference human genome annotation for The ENCODE Project
Harrow, J., Frankish, A., Gonzalez, J. M., Tapanari, E., Diekhans, M., Kokocinski, F., Aken, B. L., Barrell, D., Zadissa, A., Searle, S., et al. (2012) · Genome Research · doi:10.1101/gr.135350.111 · Cited by 4,585 · see citing works
GEOquery: a bridge between the Gene Expression Omnibus (GEO) and BioConductor
Davis, S. R., & Meltzer, P. S. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm254 · Cited by 3,287 · see citing works
Leinonen, R., Sugawara, H., & Shumway, M. F. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1019 · Cited by 2,760 · see citing works
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420 · see citing works
Margolin, A. A., Nemenman, I. M., Basso, K., Wiggins, C. H., Stolovitzky, G. A., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.7916/d8g15zbp · Cited by 2,260 · see citing works
Transcriptomic Analysis of Autistic Brain Reveals Convergent Molecular Pathology
Voineagu, I., Wang, X., Johnston, P., Lowe, J. K., Tian, Y., Horvath, S., Mill, J., Cantor, R. M., Blencowe, B. J., & Geschwind, D. H. (2011) · Nature · doi:10.1038/nature10110 · Cited by 1,874 · cited 1× · see citing works
“…the first co-expression analyses, there are studies which are highly targeted toward conditions of interest with networks derived from relatively small numbers of samples (dozens to the low hundreds) and often focusing on broad network changes in some condition of interest (Voineagu et al. , 2011).”
BackgroundReactome: a database of reactions, pathways and biological processes
Croft, D., O'Kelly, G., Wu, G., Haw, R., Gillespie, M., Matthews, L., Caudy, M., Garapati, P., Gopinath, G., Jassal, B., et al. (2011) · Nucleic Acids Research · doi:10.1093/nar/gkq1018 · Cited by 1,751 · see citing works
Online Passive-Aggressive Algorithms
Crammer, K., Dekel, O., Keshet, J., Shalev‐Shwartz, S., & Singer, Y. (2003) · Cited by 1,698 · see citing works
Faith, J. J., Hayete, B., Thaden, J. T., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, S., Collins, J. J., & Gardner, T. S. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050008 · Cited by 1,563 · see citing works
Solving large scale linear prediction problems using stochastic gradient descent algorithms
Zhang, T. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015332 · Cited by 1,130 · see citing works
Mostafavi, S., Ray, D., Warde-Farley, D., Grouios, C., & Morris, Q. (2008) · Genome biology · doi:10.1186/gb-2008-9-s1-s4 · Cited by 1,054 · see citing works
Integrative functional genomic analyses implicate specific molecular pathways and circuits in autism
Parikshak, N. N., Luo, R., Zhang, A., Won, H., Lowe, J. K., Chandran, V., Horvath, S., & Geschwind, D. H. (2013) · Cell · doi:10.1016/j.cell.2013.10.031 · Cited by 1,039 · see citing works
SEQC/MAQC-III Consortium (2014) · Nature Biotechnology · doi:10.1038/nbt.2957 · Cited by 983 · see citing works
Online Passive-Aggressive Algorithms
CrammerKoby, DekelOfer, KeshetJoseph, Shalev-ShwartzShai, & SingerYoram (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248566 · Cited by 914 · see citing works
Coexpression Analysis of Human Genes Across Many Microarray Data Sets
Lee, H. K., Hsu, A. K., Sajdak, J., Qin, J., & Pavlidis, P. (2004) · Genome Research · doi:10.1101/gr.1910904 · Cited by 770 · see citing works
Guilt-by-association goes global
Oliver, S. (2000) · Nature · doi:10.1038/35001165 · Cited by 557 · see citing works
RNA-seq differential expression studies: more sequence or more replication?
Liu, Y., Zhou, J., & White, K. P. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btt688 · Cited by 497 · see citing works
Statistical Design and Analysis of RNA Sequencing Data
Auer, P. L., & Doerge, R. W. (2010) · Genetics · doi:10.1534/genetics.110.114983 · Cited by 393 · see citing works
A critical assessment of Mus musculusgene function prediction using integrated genomic evidence
Peña-Castillo, L., Tasan, M., Myers, C. L., Lee, H., Joshi, T., Zhang, C., Guan, Y., Leone, M., Pagnani, A., Kim, W. K., et al. (2008) · Genome Biology · doi:10.1186/gb-2008-9-s1-s2 · Cited by 248 · see citing works
Detecting and correcting systematic variation in large-scale RNA sequencing data
Li, S., Łabaj, P. P., Zumbo, P., Sykacek, P., Shi, W., Shi, L., Phan, J., Wu, P.-Y., Wang, M., Wang, C., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.3000 · Cited by 175 · see citing works
The Impact of Multifunctional Genes on "Guilt by Association" Analysis
Gillis, J., & Pavlidis, P. (2011) · PLoS ONE · doi:10.1371/journal.pone.0017258 · Cited by 175 · see citing works
Self-citationSekhon, R. S., Briskine, R., Hirsch, C. N., Myers, C. L., Springer, N. M., Buell, C. R., de Leon, N., & Kaeppler, S. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0061005 · Cited by 163 · see citing works
ReCount: A multi-experiment resource of analysis-ready RNA-seq gene count datasets
Frazee, A. C., Langmead, B., & Leek, J. T. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-449 · Cited by 155 · see citing works
Gemma: a resource for the reuse, sharing and meta-analysis of expression profiling data
Zoubarev, A., Hamer, K. M., Keshav, K., et al. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts430 · Cited by 136 · see citing works
Self-citationUtilizing RNA-Seq data for <i>de novo</i> coexpression network inference
Iancu, O. D., Kawane, S., Bottomly, D., Searles, R., Hitzemann, R., & McWeeney, S. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts245 · Cited by 114 · see citing works
RSEQtools: a modular framework to analyze RNA-Seq data using compact, anonymized data summaries
Habegger, L., Sboner, A., Gianoulis, T. A., Rozowsky, J. S., Agarwal, A., Snyder, M. P., & Gerstein, M. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq643 · Cited by 109 · see citing works
Wren, J. D. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp290 · Cited by 95 · see citing works
Genes, behavior and next‐generation <scp>RNA</scp> sequencing
Hitzemann, R., Bottomly, D., Darakjian, P., Walter, N., Iancu, O., Searles, R., Wilmot, B., & McWeeney, S. (2013) · Genes, Brain and Behavior · doi:10.1111/gbb.12007 · Cited by 88 · see citing works
The role of indirect connections in gene networks in predicting function
Gillis, J., & Pavlidis, P. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr288 · Cited by 73 · see citing works
Self-citationMeasuring the wisdom of the crowds in network-based gene function inference
Verleyen, W., Ballouz, S., & Gillis, J. A. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu715 · Cited by 24 · see citing works
Self-citation
Flagged first, then by citations
2 of the 47 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 1,086 · see citing works
Impact of Genetic Polymorphisms on Human Immune Cell Gene Expression
Schmiedel, B. J., Singh, D., Madrigal, A., Valdovino-Gonzalez, A. G., White, B. M., Zapardiel-Gonzalo, J., Ha, B., Altay, G., Greenbaum, J. A., McVicker, G., et al. (2018) · Cell · doi:10.1016/j.cell.2018.10.022 · Cited by 867 · see citing works
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430 · see citing works
Crow, M., Paul, A., Ballouz, S., Huang, Z. J., & Gillis, J. (2018) · Nature Communications · doi:10.1038/s41467-018-03282-0 · Cited by 365 · see citing works
Self-citationCEMiTool: a Bioconductor package for performing comprehensive modular co-expression analyses
Russo, P. S. T., Ferreira, G. R., Cardozo, L. E., Bürger, M. C., Arias‐Carrasco, R., Maruyama, S. R., Hirata, T. D. C., de Lima, D. S., Passos, F. M., Fukutani, K. F., et al. (2018) · BMC Bioinformatics · doi:10.1186/s12859-018-2053-1 · Cited by 318 · see citing works
“around 20 samples (which is in accordance to previous findings [26]), indicating that the β x R(2) curve, and thus”
ResultLearning from Co-expression Networks: Possibilities and Challenges
Serin, E. A. R., Nijveen, H., Hilhorst, H., & Ligterink, W. (2016) · Frontiers in Plant Science · doi:10.3389/fpls.2016.00444 · Cited by 272 · see citing works
“RNA-seq based co-expression network construction is still in its infancy (Iancu et al., 2012; Ballouz et al., 2015) but the foreseen predominance”
BackgroundMethodThe United Kingdom Brain Expression Consortium, Botía, J. A., Vandrovcova, J., Forabosco, P., Guelfi, S., D’Sa, K., Hardy, J., Lewis, C. M., Ryten, M., & Weale, M. E. (2017) · BMC Systems Biology · doi:10.1186/s12918-017-0420-6 · Cited by 289 · see citing works
Advanced Applications of RNA Sequencing and Challenges
Han, Y., Gao, S., Muegge, K., Zhang, W., & Zhou, B. (2015) · Bioinformatics and Biology Insights · doi:10.4137/bbi.s28991 · Cited by 277 · see citing works
Wisecaver, J. H., Borowsky, A., Tzin, V., Jander, G., Kliebenstein, D. J., & Rokas, A. (2017) · The Plant Cell · doi:10.1105/tpc.17.00009 · Cited by 244 · see citing works
“…global coexpression networks that we have constructed accurately capture the spatial and temporal regulation of BGCs in response to the diverse ecological conditions plants experience, which is at least partially dependent on the number and types of the conditions sampled (Ballouz et al., 2015).”
BackgroundObayashi, T., Aoki, Y., Tadaka, S., Kagaya, Y., & Kinoshita, K. (2017) · Plant and Cell Physiology · doi:10.1093/pcp/pcx191 · Cited by 252 · see citing works
“The function scores were correlated with the logarithmic number of samples for each species (PCC = 0.44), as reported previously (Ballouz et al. 2015).”
MethodInfluentialCo-expression networks reveal the tissue-specific regulation of transcription and splicingPreprint
Saha, A., Kim, Y., Gewirtz, A. D., Jo, B., Gao, C., McDowell, I. C., Engelhardt, B. E., & Battle, A. J. (2017) · Genome Research · doi:10.1101/gr.216721.116 · Cited by 171 · see citing works
ATTED-II in 2016: A Plant Coexpression Database Towards Lineage-Specific Coexpression
Aoki, Y., Okamura, Y., Tadaka, S., Kinoshita, K., & Obayashi, T. (2015) · Plant and Cell Physiology · doi:10.1093/pcp/pcv165 · Cited by 167 · see citing works
“The quality of coexpression data is primarily based on the number of samples (Ballouz et al. 2015), and this characteristic limits application of coexpression analysis in non-model species.”
Evaluating measures of association for single-cell transcriptomics
Skinnider, M. A., Squair, J. W., & Foster, L. J. (2019) · Nature Methods · doi:10.1038/s41592-019-0372-4 · Cited by 145 · see citing works
Obayashi, T., Kagaya, Y., Aoki, Y., Tadaka, S., & Kinoshita, K. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky1155 · Cited by 138 · see citing works
“Importantly, the quality of coexpression data strongly depends on the sample size (1,2).”
BackgroundPredictability of human differential gene expression
Crow, M., Lim, N., Ballouz, S., Pavlidis, P., & Gillis, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802973116 · Cited by 117 · see citing works
“We first asked whether they formed coexpressed gene modules in a high-quality coexpression network (27) (Fig.”
BackgroundMethodSelf-citationSun, Y.-C., Chen, X., Fischer, S., Lu, S., Zhan, H., Gillis, J. A., & Zador, A. M. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00842-4 · Cited by 124 · see citing works
Self-citationCorrelation AnalyzeR: functional predictions from gene co-expression correlations
Miller, H. E., & Bishop, A. J. R. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04130-7 · Cited by 116 · see citing works
“Tissue-disease groups with fewer than 30 distinct samples were removed to limit the effects of bias from individual samples and improve the performance of co-expression calculations [17].”
BackgroundMethodObayashi, T., Hibara, H., Kagaya, Y., Aoki, Y., & Kinoshita, K. (2022) · Plant and Cell Physiology · doi:10.1093/pcp/pcac041 · Cited by 125 · see citing works
Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau
Rayaprolu, S., Higginbotham, L., Bagchi, P., Watson, C. M., Zhang, T., Levey, A. I., Rangaraju, S., & Seyfried, N. T. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-00840-3 · Cited by 118 · see citing works
Johnson, K. A., & Krishnan, A. (2022) · Genome biology · doi:10.1186/s13059-021-02568-9 · Cited by 108 · see citing works
“Supporting this hypothesis, other studies have noted an association between larger sample size and more accurate coexpression networks [18, 27] and subsequent network transformation with CLR [31].”
BackgroundResultExploiting single-cell expression to characterize co-expression replicability
Crow, M., Paul, A., Ballouz, S., Huang, Z. J., & Gillis, J. (2016) · Genome Biology · doi:10.1186/s13059-016-0964-6 · Cited by 76 · see citing works
“Our lab has done extensive testing and benchmarking of this algorithm against both more sophisticated machine learning methods, and using varying methods for network generation (e.g. partial correlation, mutual information, etc.), and found that results were robust [21, 26].”
BackgroundMethodInfluentialSelf-citationFu, C., Zhang, X., Veri, A. O., Iyer, K. R., Lash, E., Xue, A., Yan, H., Revie, N. M., Wong, C. J., Lin, Z., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-26850-3 · Cited by 82 · see citing works
“02 rev.0) against the C . albicans RefSeq database (version 68), as previously described 90,91 .”
MethodIntegrating gene regulatory pathways into differential network analysis of gene expression data
Grimes, T., Potter, S. S., & Datta, S. (2019) · Scientific Reports · doi:10.1038/s41598-019-41918-3 · Cited by 77 · see citing works
“This is often referred to as the guilt-by-association principle, and it provides support for the use of gene expression data in reconstructing the underlying gene-gene association network 3,4 .”
MethodShahan, R., Zawora, C., Wight, H., Sittmann, J., Wang, W., Mount, S. M., & Liu, Z. (2018) · PLANT PHYSIOLOGY · doi:10.1104/pp.18.00086 · Cited by 77 · see citing works
“Our use of multiple, comprehensive RNA-seq libraries provides extensive gene expression information for network construction and therefore increases the likelihood of identifying genetic correlations (Lee et al., 2004; Wren, 2009; Ballouz et al., 2015).”
MethodstandR: spatial transcriptomic analysis for GeoMx DSP data
Liu, N., Bhuva, D. D., Mohamed, A., Bokelund, M., Kulasinghe, A., Tan, C. W., & Davis, M. J. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad1026 · Cited by 73 · see citing works
Meta Gene Regulatory Networks in Maize Highlight Functionally Relevant Regulatory Interactions
Zhou, P., Li, Z., Ellison, E. L., Gómez-Cano, F., Crisp, P. A., Noshay, J. M., Erich Grotewold, A., Hirsch, C. N., Briggs, S. P., & Springer, N. M. (2020) · The Plant Cell · doi:10.1105/tpc.20.00080 · Cited by 71 · see citing works
“Increasing sample size was reported to generally have a positive effect on network performance (Ballouz et al., 2015; Huang et al., 2017), consistent with our comparison of tissue-specific or genotype-specific (i.e., developmental atlas) networks of different sizes.”
ResultMulti‐omics network‐based functional annotation of unknown Arabidopsis genes
Depuydt, T., & Vandepoele, K. (2021) · The Plant Journal · doi:10.1111/tpj.15507 · Cited by 73 · see citing works
(Differential) Co-Expression Analysis of Gene Expression: A Survey of Best Practices
Chowdhury, H. A., Bhattacharyya, D. K., & Kalita, J. K. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2893170 · Cited by 75 · see citing works
“b) More than 10 million reads per sample are ideal for co-expression analysis [45].”
BackgroundMethodInfluentialLiesecke, F., Daudu, D., de Bernonville, R. D., Besseau, S., Clastre, M., Courdavault, V., Craene, J. D., Crèche, J., Giglioli‐Guivarc’h, N., Glévarec, G., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-29077-3 · Cited by 70 · see citing works
Big data: Historic advances and emerging trends in biomedical research
Cremin, C. J., Dash, S., & Huang, X. (2022) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2022.02.004 · Cited by 80 · see citing works
Kim, M., Haney, J. R., Zhang, P., Hernandez, L. M., Wang, L.-K., Pérez‐Cano, L., Loohuis, L. M. O., de la Torre-Ubieta, L., & Gandal, M. J. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00847-z · Cited by 74 · see citing works
“Even after stratifying samples by imputed C4A copy number, this sample size was sufficient 82,83 to detect significant gene co-expression, as we observed.”
B chromosomes: from cytogenetics to systems biology
(2016) · doi:10.1007/s00412-016-0613-6 · Cited by 70 · see citing works
“The inference of coexpression networks (Spearman’s rank correlation) (Ballouz et al. 2015) using RNA-Seq from the B+ and B − genotypes of A.”
MethodOvens, K. L., Eames, B. F., & McQuillan, I. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.695399 · Cited by 65 · see citing works
“Another possible limitation is the sample size of the dataset used to construct a network (Ballouz et al., 2015; Ovens et al., 2020) and finding multi-species studies to make evolutionary inferences.”
BackgroundMethodUtility and Limitations of Using Gene Expression Data to Identify Functional Associations
Uygun, S., Peng, C., Lehti‐Shiu, M. D., Last, R. L., & Shiu, S. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005244 · Cited by 62 · see citing works
Transcriptomic Insight Into the Polygenic Mechanisms Underlying Psychiatric Disorders
Hernandez, L. M., Kim, M., Hoftman, G. D., Haney, J. R., de la Torre-Ubieta, L., Paşaniuc, B., & Gandal, M. J. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.06.005 · Cited by 57 · see citing works
“gene expression variance explained by each covariate was calculated using a linear mixed effects model using the variancePartition package (28).”
MethodConstruction and Optimization of a Large Gene Coexpression Network in Maize Using RNA-Seq Data
Huang, J., Vendramin, S., Shi, L., & McGinnis, K. M. (2017) · PLANT PHYSIOLOGY · doi:10.1104/pp.17.00825 · Cited by 51 · see citing works
“…Pearson correlation coef fi cient (PCC), Spearman ’ s correlation coef fi cient (SCC), Kendall rank correlation co-ef fi cient (KCC), Gini correlation coef fi cient (GCC), and biweight midcorrelation (BIC; Langfelder and Horvath, 2008; Kumari et al., 2012; Ma and Wang, 2012; Ballouz et al., 2015).”
BackgroundMethodResultInfluentialMistry, D., Wise, R. P., & Dickerson, J. A. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187091 · Cited by 51 · see citing works
Liesecke, F., Craene, J. D., Besseau, S., Courdavault, V., Clastre, M., Vergès, V., Papon, N., Giglioli‐Guivarc’h, N., Glévarec, G., Pichon, O., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-50885-8 · Cited by 44 · see citing works
CoCoCoNet: conserved and comparative co-expression across a diverse set of species
Lee, J., Shah, M., Ballouz, S., Crow, M., & Gillis, J. A. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa348 · Cited by 38 · see citing works
Self-citationMachine Learning-Based State-of-the-Art Methods for the Classification of RNA-Seq Data
Jabeen, A., Ahmad, N., & Raza, K. (2017) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-3-319-65981-7_6 · Cited by 57 · see citing works
Systems Biology Approaches for Host–Fungal Interactions: An Expanding Multi-Omics Frontier
Culibrk, L., Croft, C. A., & Tebbutt, S. J. (2016) · OMICS A Journal of Integrative Biology · doi:10.1089/omi.2015.0185 · Cited by 48 · see citing works
“Despite this, current methods are still unable to provide a definitive model of gene regulatory networks (Ballouz et al., 2015).”
MethodLSTrAP: efficiently combining RNA sequencing data into co-expression networks
Proost, S., Krawczyk, A., & Mutwil, M. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1861-z · Cited by 38 · see citing works
“While the bare minimum was estimated to be >20 samples with >10 million reads, more samples and higher read depth were found to increase the predictive capabilities of the resulting network [14].”
BackgroundMethodCoexpression reveals conserved gene programs that co-vary with cell type across kingdoms
Crow, M., Suresh, H., Lee, J., & Gillis, J. A. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac276 · Cited by 36 · see citing works
Self-citationPark, Y., Heider, D., & Hauschild, A.-C. (2021) · Cancers · doi:10.3390/cancers13133148 · Cited by 41 · see citing works
“compared microarray and NGS-based co-expression networks and found the existence of a bias originating from batch effects between the two technologies [80].”
BackgroundDeane, C. S., Willis, C. R. G., Phillips, B. E., Atherton, P. J., Harries, L. W., Ames, R. M., Szewczyk, N. J., & Etheridge, T. (2021) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.12706 · Cited by 40 · see citing works
Liang, Y., & Kelemen, A. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx036 · Cited by 35 · see citing works
Hin, N., Newman, M., Kaslin, J., Douek, A. M., Lumsden, A. L., Nik, S. H. M., Dong, Y., Zhou, X., Manucat-Tan, N. B., Ludington, A. J., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227258 · Cited by 40 · see citing works
“Although revealing valuable insights, our comparison of the gene co-expression patterns in the zebrafish and human datasets is limited by inherent differences in species-level gene expression, differences in the brain regions and tissues sampled in each dataset, and difference in the RNA-seq platforms used to collect…”
BackgroundMethodYlla, G., Piulachs, M., & Bellés, X. (2017) · BMC Genomics · doi:10.1186/s12864-017-4177-5 · Cited by 38 · see citing works
Genomic data integration systematically biases interactome mapping
Skinnider, M. A., Stacey, R. G., & Foster, L. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006474 · Cited by 35 · see citing works
“We began by analyzing the functional connectivity of each network, defined as the degree to which the function of any given protein can be predicted from those of its interacting partners, based on the principle of ‘guilt by association’ [21,22].”
MethodEmamjomeh, A., Robat, E. S., Zahiri, J., Solouki, M., & Khosravi, P. (2017) · Plant Biotechnology Reports · doi:10.1007/s11816-017-0433-z · Cited by 38 · see citing works
“Algorithms based on RNA-Seq Although DNA microarray has been the main platform for expression data, recently, RNA-Seq has played an important role in quantitative analysis of gene expression and detection of transcripts (Richard et al. 2010; Giorgi et al. 2013; Ballouz et al. 2015).”
MethodCo-expression of calcium and hERG potassium channels reduces the incidence of proarrhythmic events
Ballouz, S., Mangala, M. M., Perry, M., Heitmann, S., Gillis, J. A., Hill, A. P., & Vandenberg, J. I. (2020) · Cardiovascular Research · doi:10.1093/cvr/cvaa280 · Cited by 32 · see citing works
Self-citationStem cells and heart disease - Brake or accelerator?
Steinhoff, G., Nesteruk, J., Wolfien, M., Große, J., Ruch, U., Vasudevan, P., & Müller, P. (2017) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2017.10.007 · Cited by 36 · see citing works
CoPhosK: A method for comprehensive kinase substrate annotation using co-phosphorylation analysis
Ayati, M., Wiredja, D. D., Schlatzer, D., Maxwell, S., Li, M., Koyutürk, M., & Chance, M. R. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006678 · Cited by 30 · see citing works
Yang, S., Marín‐Juez, R., Meijer, A. H., & Spaink, H. P. (2015) · BMC Genomics · doi:10.1186/s12864-015-1740-9 · Cited by 33 · see citing works
Obayashi, T., Kodate, S., Hibara, H., Kagaya, Y., & Kinoshita, K. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac983 · Cited by 31 · see citing works
Aono, A. H., Pimenta, R. J. G., Garcia, A. L. B., Correr, F. H., Hosaka, G. K., Carrasco, M. M., Cardoso-Silva, C. B., Mancini, M. C., Sforça, D. A., Santos, L. B. D., et al. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2021.668623 · Cited by 29 · see citing works
“Pearson correlation coefficients were used because of their reasonable performance in RNA-Seq datasets (Ballouz et al., 2015).”
MethodYang, L., Su, Z., Wang, Z., Li, Z., Shang, Z., Du, H., Liu, G., Qi, D., Yang, Z., Xu, Z., et al. (2021) · Cell Death and Disease · doi:10.1038/s41419-021-03552-8 · Cited by 36 · see citing works
Meng, G., & Mei, H. (2019) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2019.00101 · Cited by 28 · see citing works
(2017) · doi:10.1104/pp.16.01982 · Cited by 29 · see citing works
“The pathogen is soil borne and remains in infected roots, where it produces toxins that cause the foliar SDS (Brar et al., 2011; Brar and Bhattacharyya, 2012; Pudake et al., 2013; Chang et al., 2016).”
BackgroundNetwork aggregation improves gene function prediction of grapevine gene co-expression networks
Wong, D. C. J. (2020) · Plant Molecular Biology · doi:10.1007/s11103-020-01001-2 · Cited by 28 · see citing works
Brain transcriptome atlases: a computational perspective
Mahfouz, A. E., Huisman, S. M. H., Lelieveldt, B. P. F., & Reinders, M. J. T. (2016) · Brain Structure and Function · doi:10.1007/s00429-016-1338-2 · Cited by 26 · see citing works
“There are numerous technical considerations to considere while constructing co-expression networks that go beyond the scope of this review (Allen et al. 2012; Ballouz et al. 2015).”
BackgroundCellMapper: rapid and accurate inference of gene expression in difficult-to-isolate cell types
Nelms, B. D., Waldron, L., Barrera, L. A., Weflen, A. W., Goettel, J. A., Guo, G., Montgomery, R. K., Neutra, M. R., Breault, D. T., Snapper, S. B., et al. (2016) · Genome biology · doi:10.1186/s13059-016-1062-5 · Cited by 24 · see citing works
“Many algorithms that explore gene co-expression relationships have translated well to RNA sequencing (RNA-Seq) data [46], and CellMapper in principle could be adapted for RNA-Seq to allow for more complete coverage of the transcriptome.”
BackgroundSun, H., Zhao, K., Zhou, M., Chen, Y., & Guan, L. L. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty883 · Cited by 25 · see citing works
DeORFanizing Candida albicans Genes using Coexpression
O’Meara, T. R., & O’Meara, M. J. (2021) · mSphere · doi:10.1128/msphere.01245-20 · Cited by 20 · see citing works
“Genome scale coexpression networks built from bulk RNAseq studies have been used extensively for gene function prediction using guilt by association (1, 2, 48).”
Construction, comparison and evolution of networks in life sciences and other disciplines
Gysi, D. M., & Nowick, K. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2019.0610 · Cited by 25 · see citing works
“They are best built from genome-wide expression data measured by RNA-Seq [124] or microarrays [125,126].”
BackgroundDifferential Regulatory Analysis Based on Coexpression Network in Cancer Research
Li, J., Li, Y., & Li, Y. (2016) · BioMed Research International · doi:10.1155/2016/4241293 · Cited by 23 · see citing works
“suggest minimal experimental criteria to obtain useful functional connectivity and topology information of coexpression with microarrays greater than 20 samples and read depth greater than 10M per sample [64].”
BackgroundPhytoNet: comparative co-expression network analyses across phytoplankton and land plants
Ferrari, C., Proost, S., Ruprecht, C., et al. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky298 · Cited by 24 · see citing works
Qian, Y.-Q., Zhou, Z.-W., Ouyang, T., Li, D., Li, R., Gan, P., Qiao, R., Tan, Y. L., Qian, M., Liu, L., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62887-4 · Cited by 27 · see citing works
“Four tools were used to construct gene and TE co-expression networks: WGCNA 125 , Pearson correlation coef fi cient (PCC) 126 – 128 , CORNET 129 , and TEffectR 130 .”
InfluentialNoise regularization removes correlation artifacts in single-cell RNA-seq data preprocessing
Zhang, R., Atwal, G. S., & Lim, W. K. (2021) · Patterns · doi:10.1016/j.patter.2021.100211 · Cited by 21 · see citing works
Pang, C. N. I., Ballouz, S., Weissberger, D., Thibaut, L. M., Hamey, J. J., Gillis, J. A., Wilkins, M. R., & Hart‐Smith, G. (2020) · Molecular & Cellular Proteomics · doi:10.1074/mcp.ra120.002154 · Cited by 18 · see citing works
“For merged networks, gene co-expression networks for yeast (47) or human (48), or high confidence PPI networks for yeast (44) or human (45), were merged with CF-MS networks.”
BackgroundMethodSelf-citationKosvyra, A., Maramis, C., & Chouvarda, I. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103971 · Cited by 15 · see citing works
Breen, M. S., Stein, D. J., & Baldwin, D. S. (2016) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2546 · Cited by 18 · see citing works
“To obtain gene module reproducibility under a case–control design, it is recommended to construct networks using a minimum of 20 independent samples per condition and to perform permutation analyses on modules of interest to provide in silico support of coexpression (Ballouz et al., 2015).”
MethodVandenbon, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263344 · Cited by 20 · see citing works
“High numbers of samples are needed to accurately infer correlation of expression [11].”
BackgroundResultInfluentialKober, K. M., Schumacher, M. A., Conley, Y. P., Topp, K. S., Mazor, M. B., Hammer, M. J., Paul, S. M., Levine, J. D., & Miaskowski, C. A. (2019) · Molecular Pain · doi:10.1177/1744806919878088 · Cited by 19 · see citing works
“Although the performance of co-expression network analysis is dependent of sample size, WGCNA performs well on sample sizes of >20.(52,53) Multiple logistic regression analysis, which controlled for significant demographic (i.”
MethodSystems biology and gene networks in Alzheimer’s disease
Wang, Z.-T., Tan, C., Tan, L., & Yu, J. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.11.007 · Cited by 20 · see citing works
“network, and the use of statistical modeling and multiple corrections (Ballouz et al., 2015; Contreras-Lopez et al., 2018; Gaiteri et al., 2014; Liu et al., 2014; Rapaport et al., 2013; Robles et al., 2012; Song et al., 2012).”
Fortelny, N., Butler, G. S., Overall, C. M., & Pavlidis, P. (2017) · Molecular & Cellular Proteomics · doi:10.1074/mcp.m116.065706 · Cited by 19 · see citing works
“Finally, with the aim of deriving a more robust coexpression result, we performed a meta-analysis (34, 35) of gene expression over multiple microarray datasets (listed in supplemental Table S6 and shown in Fig.”
O’Meara, M. J., Ballouz, S., Shoichet, B. K., & Gillis, J. A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0160098 · Cited by 18 · see citing works
“This improvement is particularly striking as the performance of the CoExp network has diminishing returns as sample size grows [29] (Fig 4B), suggesting that no reasonable amount of new expression data could compensate for the medicinal chemistry data captured by SEA.”
BackgroundSelf-citationPiya, S., Hawk, T. E., Patel, B., Baldwin, L., Rice, J. H., Stewart, C. N., & Hewezi, T. (2021) · Molecular Plant Pathology · doi:10.1111/mpp.13168 · Cited by 16 · see citing works
Li, Z., Lou, Y., Tian, G., Wu, J., Lu, A., Chen, J., Xu, B., Shi, J., & Yang, J. (2019) · Aging · doi:10.18632/aging.102579 · Cited by 18 · see citing works
“Streamlining the analysis of multiple datasets using the same computational processes enhances statistical power and may significantly increase the accuracy of the findings [41, 42].”
MethodGene communities in co-expression networks across different tissues
Russell, M., Aqil, A., Saitou, M., Gökçümen, Ö., & Masuda, N. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011616 · Cited by 17 · see citing works
“Systematically investigating biological performance of network community detection as a function of the number of samples [135, 137, 138] will help us to better understand potentials and limitations of both single-layer and multilayer community detection in gene and other related networks, which is…”
PlantNexus: A Gene Co-expression Network Database and Visualization Tool for Barley and Sorghum
Zhou, Y., Sukul, A., Mishler‐Elmore, J. W., Faik, A., & Held, M. (2022) · Plant and Cell Physiology · doi:10.1093/pcp/pcac007 · Cited by 16 · see citing works
GCEN: An Easy-to-Use Toolkit for Gene Co-Expression Network Analysis and lncRNAs Annotation
Chen, W., Li, J., Huang, S., Li, X., Zhang, X., Hu, X., Xiang, S., & Liu, C. (2022) · Current Issues in Molecular Biology · doi:10.3390/cimb44040100 · Cited by 16 · see citing works
“figured out that network aggregation shows better performance [11,27].”
BackgroundMethodInfluentialMachine Learning-based state-of-the-art methods for the classification of RNA-Seq data
(2017) · doi:10.1101/120592 · Cited by 18 · see citing works
“K.”
Guo, W., Calixto, C. P. G., Τzioutziou, N. A., Lin, P., Waugh, R., Brown, J. W. S., & Zhang, R. (2017) · BMC Systems Biology · doi:10.1186/s12918-017-0440-2 · Cited by 18 · see citing works
Co-expression networks reveal the tissue-specific regulation of transcription and splicingPreprint
Saha, A., Kim, Y., Gewirtz, A. D., Jo, B., Gao, C., McDowell, I. C., Engelhardt, B. E., & Battle, A. J. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/078741 · Cited by 16 · see citing works
“Many regulatory relationships are shared between tissues, and assessing hubs across all tissues jointly may improve robustness (Ballouz et al. 2015).”
Addressing noise in co-expression network construction
Burns, J., Shealy, B. T., Greer, M. S., Hadish, J. A., McGowan, M., Biggs, T. D., Smith, M. C., Feltus, F. A., & Ficklin, S. P. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab495 · Cited by 17 · see citing works
Abbassi‐Daloii, T., Kan, H. E., Raz, V., & Hoen, P. A. C. ’. (2020) · Genomics · doi:10.1016/j.ygeno.2020.05.026 · Cited by 14 · see citing works
The transcriptional legacy of developmental stochasticity
(2019) · doi:10.1101/2019.12.11.873265 · Cited by 16 · see citing works
“For each quadruplet at each time point, we constructed a co-expression network(27).”
MethodSelf-citationEmad, A., & Sinha, S. (2018) · npj Systems Biology and Applications · doi:10.1038/s41540-021-00169-7 · Cited by 15 · see citing works
Ebner, F., Kuhring, M., Radonić, A., Midha, A., Renard, B. Y., & Hartmann, S. (2018) · Frontiers in Immunology · doi:10.3389/fimmu.2018.01868 · Cited by 14 · see citing works
“We applied established and efficient procedures to infer and analyze GENs including pairwise correlation, cluster (module) analysis, and functional enrichment (95, 96).”
MethodOmics approaches to study gene regulatory networks for development in echinoderms
Lowe, E. K., Cuomo, C., & Arnone, M. I. (2017) · Briefings in Functional Genomics · doi:10.1093/bfgp/elx012 · Cited by 15 · see citing works
Metabolic engineering to increase crop yield: From concept to execution
Skraly, F. A., Ambavaram, M. M. R., Peoples, O. P., & Snell, K. D. (2018) · Plant Science · doi:10.1016/j.plantsci.2018.03.011 · Cited by 15 · see citing works
Positive and negative forms of replicability in gene network analysis
Verleyen, W., Ballouz, S., & Gillis, J. A. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv734 · Cited by 12 · see citing works
Self-citationCan dynamic network modelling be used to identify adaptive microbiomes?
Garcia, J., & Kao‐Kniffin, J. (2019) · Functional Ecology · doi:10.1111/1365-2435.13491 · Cited by 13 · see citing works
Gene Differential Co-Expression Networks Based on RNA-Seq: Construction and Its Applications
Wang, P., & Wang, D. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3103280 · Cited by 14 · see citing works
“As an alternative of GCNs, differential rewiring or differential co-expression among genes has been proved to be an effective tool to identify informative genes that may contribute to the phenotype differences between treated and con-trol samples [27], [43], [44], [45], [46], [47], [48], [49].”
MethodInfluentialDTW-MIC Coexpression Networks from Time-Course Data
Riccadonna, S., Jurman, G., Visintainer, R., Filosi, M., & Furlanello, C. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152648 · Cited by 17 · see citing works
Orduña, L., Santiago, A., Navarro‐Payá, D., Zhang, C., Wong, D. C. J., & Matus, J. T. (2023) · Journal of Experimental Botany · doi:10.1093/jxb/erad344 · Cited by 13 · see citing works
- Authors
- Ballouz, S., Verleyen, W., & Gillis, J.
- Affiliations
- Cold Spring Harbor Laboratory (US)
- Corresponding author
- Ballouz, S.
- Gillis, J.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-02-28
- Volume
- 31
- Issue
- 13
- Pages
- 2123-2130
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2104724027
- Cited by
- 215
- References
- 47
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 7 · 2023: 21 · 2022: 15 · 2021: 31 · 2020: 18 · 2019: 27 · 2018: 24 · 2017: 23 · 2016: 20 · 2015: 6 · 2014: 1
- FWCI
- 10.55
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) (acceptedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- guilt by association, gene co-expression, network topology, hub genes, sample size requirements, sequencing depth, stochastic gene expression, microarray comparison, functional connectivity, gene function prediction
- MeSH terms
- Algorithms, Brain, Humans, Models, Theoretical, Sequence Analysis, RNA, Computational Biology, Gene Expression Profiling, Gene Regulatory Networks, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
