Comparative assessment of differential network analysis methods
Abstract
Differential network analysis (DiNA) denotes a recent class of network-based Bioinformatics algorithms which focus on the differences in network topologies between two states of a cell, such as healthy and disease, to identify key players in the discriminating biological processes. In contrast to conventional differential analysis, DiNA identifies changes in the interplay between molecules, rather than changes in single molecules. This ability is especially important in cases where effectors are changed, e.g. mutated, but their expression is not. A number of different DiNA approaches have been proposed, yet a comparative assessment of their performance in different settings is still lacking. In this paper, we evaluate 10 different DiNA algorithms regarding their ability to recover genetic key players from transcriptome data. We construct high-quality regulatory networks and enrich them with co-expression data from four different types of cancer. Next, we assess the results of applying DiNA algorithms on these data sets using a gold standard list (GSL). We find that local DiNA algorithms are generally superior to global algorithms, and that all DiNA algorithms outperform conventional differential expression analysis. We also assess the ability of DiNA methods to exploit additional knowledge in the underlying cellular networks. To this end, we enrich the cancer-type specific networks with known regulatory miRNAs and compare the algorithms performance in networks with and without miRNA. We find that including miRNAs consistently and considerably improves the performance of almost all tested algorithms. Our results underline the advantages of comprehensive cell models for the analysis of -omics data.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It remains difficult to evaluate differential co-expression algorithms owing to the lack of gold standard gene sets to validate the outcome of these methods.
“An attempt was recently made to compare 10 differential co-expression algorithms, but concluded that it remains difficult to evaluate these owing to the lack of gold standard gene sets to validate the outcome of these methods [143].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [143]
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
WGCNA: an R package for weighted correlation network analysis
Langfelder, P., & Horvath, S. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-559 · Cited by 27,252 · see citing works
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480 · see citing works
Differential expression analysis for sequence count data
Anders, S., & Huber, W. (2010) · Genome Biology · doi:10.1186/gb-2010-11-10-r106 · Cited by 15,581 · see citing works
Lewis, B. P., Burge, C. B., & Bartel, D. P. (2005) · Cell · doi:10.1016/j.cell.2004.12.035 · Cited by 11,855 · see citing works
Vogelstein, B., Papadopoulos, N., Velculescu, V. E., Zhou, S., Diaz, L. A., & Kinzler, K. W. (2013) · Science · doi:10.1126/science.1235122 · Cited by 7,863 · see citing works
Exploring Network Structure, Dynamics, and Function Using NetworkX
Hagberg, A. A., Schult, D. A., & Swart, P. J. (2008) · Proceedings of the Python in Science Conference · doi:10.25080/tcwv9851 · Cited by 7,477 · see citing works
Oncomirs — microRNAs with a role in cancer
Esquela‐Kerscher, A., & Slack, F. J. (2006) · Nature reviews. Cancer · doi:10.1038/nrc1840 · Cited by 7,038 · see citing works
Newman, M. (2010) · doi:10.1093/acprof:oso/9780199206650.001.0001 · Cited by 6,595 · see citing works
Mammalian microRNAs predominantly act to decrease target mRNA levels
Guo, H., Ingolia, N. T., Weissman, J. S., & Bartel, D. P. (2010) · Nature · doi:10.1038/nature09267 · Cited by 3,886 · see citing works
John, B., Enright, A. J., Aravin, A., Tuschl, T., Sander, C., & Marks, D. S. (2004) · PLoS Biology · doi:10.1371/journal.pbio.0020363 · Cited by 3,796 · see citing works
Differential analysis of gene regulation at transcript resolution with RNA-seq
Trapnell, C., Hendrickson, D. G., Sauvageau, M., Goff, L., Rinn, J. L., & Pachter, L. (2013) · Nature Biotechnology · doi:10.1038/nbt.2450 · Cited by 3,492 · see citing works
On the Interpretation of χ 2 from Contingency Tables, and the Calculation of P
Fisher, R. A. (1922) · Journal Of The Royal Statistical Society · doi:10.2307/2340521 · Cited by 2,975 · see citing works
MINIMUM REDUNDANCY FEATURE SELECTION FROM MICROARRAY GENE EXPRESSION DATA
Ding, C. H. Q., & PENG, H. (2005) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720005001004 · Cited by 2,775 · see citing works
Inferring Correlation Networks from Genomic Survey Data
Friedman, J., & Alm, E. J. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002687 · Cited by 2,688 · see citing works
TRANSFAC(R) and its module TRANSCompel(R): transcriptional gene regulation in eukaryotes
Matys, V. (2005) · Nucleic Acids Research · doi:10.1093/nar/gkj143 · Cited by 2,179 · see citing works
Comparison of Values of Pearson's and Spearman's Correlation Coefficients on the Same Sets of Data
Hauke, J., & Kossowski, T. M. (2011) · Quaestiones Geographicae · doi:10.2478/v10117-011-0021-1 · Cited by 2,013 · see citing works
miR2Disease: a manually curated database for microRNA deregulation in human disease
Jiang, Q., Wang, Y., Hao, Y., Juan, L., Teng, M., Zhang, X., Li, M., Wang, G., & Liu, Y. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn714 · Cited by 1,425 · see citing works
miRTarBase: a database curates experimentally validated microRNA–target interactions
Hsu, S.-D., Lin, F., Wu, W.-Y., Liang, C., Huang, W.-C., Chan, W.-L., Tsai, W.-T., Chen, G.-Z., Lee, C.-J., Chiu, C.-M., et al. (2010) · Nucleic Acids Res. · doi:10.1093/nar/gkq1107 · Cited by 1,387 · see citing works
DIANA-microT web server v5.0: service integration into miRNA functional analysis workflows
Paraskevopoulou, M., Georgakilas, G., Kostoulas, N., Vlachos, I., Vergoulis, T., Reczko, M., Filippidis, C., Dalamagas, T., & Hatzigeorgiou, A. (2013) · Nucleic Acids Res. · doi:10.1093/nar/gkt393 · Cited by 1,104 · see citing works
microRNA target predictions in animals
Rajewsky, N. (2006) · Nature Genetics · doi:10.1038/ng1798 · Cited by 1,092 · see citing works
Flicek, P., Amode, M., Barrell, D., Beal, K., Billis, K., Brent, S., Carvalho-Silva, D., Clapham, P., Coates, G., Fitzgerald, S., et al. (2013) · Nucleic Acids Res. · doi:10.1093/nar/gkt1196 · Cited by 926 · see citing works
Ideker, T., & Sharan, R. (2008) · Genome Research · doi:10.1101/gr.071852.107 · Cited by 817 · see citing works
Molecular networks as sensors and drivers of common human diseases
Schadt, E. E. (2009) · Nature · doi:10.1038/nature08454 · Cited by 806 · see citing works
Ideker, T. E., & Krogan, N. J. (2012) · Molecular Systems Biology · doi:10.1038/msb.2011.99 · Cited by 784 · see citing works
MicroRNA-Mediated Feedback and Feedforward Loops Are Recurrent Network Motifs in Mammals
Tsang, J. S., Zhu, J., & van Oudenaarden, A. (2007) · Molecular Cell · doi:10.1016/j.molcel.2007.05.018 · Cited by 723 · see citing works
miRDB: A microRNA target prediction and functional annotation database with a wiki interface
Wang, X. (2008) · RNA · doi:10.1261/rna.965408 · Cited by 712 · see citing works
miRCancer: a microRNA–cancer association database constructed by text mining on literature
Xie, B., Ding, Q., Han, H., & Wu, D. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt014 · Cited by 573 · see citing works
Sethupathy, P., Megraw, M. S., & Hatzigeorgiou, A. G. (2006) · Nature Methods · doi:10.1038/nmeth954 · Cited by 540 · see citing works
Genenames.org: the HGNC resources in 2015
Gray, K. A., Yates, B., Seal, R. L., Wright, M. W., & Bruford, E. A. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1071 · Cited by 493 · see citing works
High‐Betweenness Proteins in the Yeast Protein Interaction Network
Joy, M. P., Brock, A., Ingber, D. E., & Huang, S. (2005) · BioMed Research International · doi:10.1155/jbb.2005.96 · Cited by 486 · see citing works
Common features of microRNA target prediction tools
Peterson, S. M., Thompson, J. A., Ufkin, M. L., Sathyanarayana, P., Liaw, L., & Congdon, C. B. (2014) · Frontiers in Genetics · doi:10.3389/fgene.2014.00023 · Cited by 468 · see citing works
Rewiring of Genetic Networks in Response to DNA Damage
Bandyopadhyay, S., Mehta, M., Kuo, D., Sung, M.-K., Chuang, R., Jaehnig, E. J., Bodenmiller, B., Licon, K., Copeland, W., Shales, M., et al. (2010) · Science · doi:10.1126/science.1195618 · Cited by 447 · see citing works
de la Fuente, A. (2010) · Trends in Genetics · doi:10.1016/j.tig.2010.05.001 · Cited by 420 · see citing works
TransmiR: a transcription factor–microRNA regulation database
Wang, J., Lu, M., Qiu, C., & Cui, Q. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp803 · Cited by 404 · see citing works
Weighted gene coexpression network analysis strategies applied to mouse weight
Fuller, T. F., Ghazalpour, A., Aten, J. E., Drake, T. A., Lusis, A. J., & Horvath, S. (2007) · Mammalian Genome · doi:10.1007/s00335-007-9043-3 · Cited by 374 · see citing works
Dirk Koschützki, A., & Schreiber, F. (2008) · Gene Regulation and Systems Biology · doi:10.4137/grsb.s702 · Cited by 344 · see citing works
Carter, S. L., Brechbühler, C. M., Griffin, M., & Bond, A. T. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth234 · Cited by 344 · see citing works
Chapter 5: Network Biology Approach to Complex Diseases
Cho, D.-Y., Kim, Y.-A., & Przytycka, T. M. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002820 · Cited by 283 · see citing works
MalaCards: an integrated compendium for diseases and their annotation
Rappaport, N., Nativ, N., Stelzer, G., Twik, M., Guan‐Golan, Y., Stein, T. I., Bahir, I., Belinky, F., Morrey, C. P., Safran, M., et al. (2013) · Database · doi:10.1093/database/bat018 · Cited by 280 · see citing works
Differential coexpression analysis using microarray data and its application to human cancer
Choi, J. K., Yu, U., Yoo, O. J., & Kim, S. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti722 · Cited by 265 · see citing works
ORegAnno: an open-access community-driven resource for regulatory annotation
Griffith, O. L., Montgomery, S. B., Bernier, B., Chu, B., Kasaian, K., Aerts, S., Mahony, S., Sleumer, M. C., Bilenky, M., Haeussler, M., et al. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm967 · Cited by 233 · see citing works
Zhang, H., Kuang, S., Xiong, X., Gao, T., Liu, C., & Guo, A.-Y. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbt085 · Cited by 222 · see citing works
Hudson, N. J., Reverter, A., & Dalrymple, B. P. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000382 · Cited by 221 · see citing works
Winter, C., Kristiansen, G., Kersting, S., Roy, J., Aust, D. E., Knösel, T., Rümmele, P., Jahnke, B., Hentrich, V., Rückert, F., et al. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002511 · Cited by 210 · see citing works
Lim, W. K., Wang, K., Lefèbvre, C., & Califano, A. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm201 · Cited by 190 · see citing works
Diseases as network perturbations
del Sol, A., Balling, R., Hood, L., & Galas, D. J. (2010) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2010.07.010 · Cited by 184 · see citing works
A statistical method for identifying differential gene–gene co-expression patterns
Lai, Y., Wu, B., Chen, L., & Zhao, H. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth379 · Cited by 167 · see citing works
DiffCorr: An R package to analyze and visualize differential correlations in biological networks
Fukushima, A. (2012) · Gene · doi:10.1016/j.gene.2012.11.028 · Cited by 165 · see citing works
Pawson, T. J., & Linding, R. (2008) · FEBS Letters · doi:10.1016/j.febslet.2008.02.011 · Cited by 163 · see citing works
Differential analysis of DNA microarray gene expression data
Hatfield, G. W., Hung, S., & Baldi, P. (2003) · Molecular Microbiology · doi:10.1046/j.1365-2958.2003.03298.x · Cited by 130 · see citing works
Ye, H., Liu, X., Lv, M., Wu, Y., Kuang, S., Gong, J., Yuan, P., Zhong, Z., Li, Q., Jia, H., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks175 · Cited by 127 · see citing works
Illuminating drug discovery with biological pathways
Apic, G., Ignjatovic, T., Boyer, S. K., & Russell, R. B. (2005) · FEBS Letters · doi:10.1016/j.febslet.2005.02.023 · Cited by 112 · see citing works
Cheng, C., Yan, K., Hwang, W., Qian, J. X., Bhardwaj, N., Rozowsky, J. S., Lu, Z. J., Niu, W., Alves, P., Kato, M., et al. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002190 · Cited by 100 · see citing works
How MicroRNA and Transcription Factor Co-regulatory Networks Affect Osteosarcoma Cell Proliferation
Poos, K., Smida, J., Nathrath, M., Maugg, D., Baumhoer, D., & Korsching, E. (2013) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1003210 · Cited by 74 · see citing works
Su, W., Kleinhanz, R. R., & Schadt, E. E. (2011) · Molecular Systems Biology · doi:10.1038/msb.2011.23 · Cited by 71 · see citing works
Transcription Regulatory Regions Database (TRRD): its status in 2000
Kolchanov, N. A. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.298 · Cited by 56 · see citing works
Ranking of network elements based on functional substructures
Dirk Koschützki, A., Schwöbbermeyer, H., & Schreiber, F. (2007) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2007.05.038 · Cited by 43 · see citing works
Essential Proteins Discovery from Weighted Protein Interaction Networks
Li, M., Wang, J., Wang, H., et al. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13078-6_11 · Cited by 42 · see citing works
New network topology approaches reveal differential correlation patterns in breast cancer
Bockmayr, M. L., Klauschen, F., Győrffy, B., Denkert, C., & Budczies, J. (2013) · BMC Systems Biology · doi:10.1186/1752-0509-7-78 · Cited by 39 · see citing works
A New Method for Identifying Essential Proteins Based on Edge Clustering Coefficient
Wang, H., Li, M., Wang, J., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21260-4_12 · Cited by 36 · see citing works
Regulation patterns in signaling networks of cancer
Schramm, G., Nandakumar, K., & König, R. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-162 · Cited by 35 · see citing works
Essential Protein Discovery Based on Network Motif and Gene Ontology
Kim, W., Li, M., Wang, J., et al. (2011) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2011.46 · Cited by 31 · see citing works
Network analysis of microRNAs and their regulation in human ovarian cancer
Schmeier, S., Schaefer, U., Essack, M., et al. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-183 · Cited by 23 · see citing works
Computer-assisted curation of a human regulatory core network from the biological literature
Thomas, P. E., Durek, P., Solt, I., Klinger, B., Witzel, F., Schulthess, P., Mayer, Y., Tikk, D., Blüthgen, N., & Leser, U. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu795 · Cited by 10 · see citing works
Self-citationInference of Surface Membrane Factors of HIV-1 Infection through Functional Interaction Networks
Jaeger, S., Ertaylan, G., van Dijk, D., Leser, U., & Sloot, P. M. A. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013139 · Cited by 8 · see citing works
Self-citationLinkages of nuclear architecture to biological and pathological control of gene expression
Stein, G. S., van Wijnen, A. J., Stein, J. L., Lian, J. B., Pockwinse, S. M., & McNeil, S. M. (1998) · Journal of Cellular Biochemistry · doi:10.1002/(sici)1097-4644(1998)72:30/31+ · Cited by 6 · see citing works
Ranking differential genes in co-expression networks
Odibat, O., & Reddy, C. K. A. (2011) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/2147805.2147849 · Cited by 4 · see citing works
Johnson, P. (2007) · Statistical Methods in Medical Research · doi:10.1177/09622802070160010602 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 1,086 · see citing works
- Authors
- Lichtblau, Y., Zimmermann, K., Haldemann, B., Lenze, D., Hummel, M., & Leser, U.
- Affiliations
- Humboldt-Universität zu Berlin (DE)
- Charité – Universitätsmedizin Berlin (DE)
- Corresponding author
- Lichtblau, Yvonne
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-07-29
- Volume
- 18
- Issue
- 5
- Pages
- bbw061
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W2494631976
- Cited by
- 76
- References
- 69
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 6 · 2023: 8 · 2022: 7 · 2021: 13 · 2020: 19 · 2019: 9 · 2018: 2 · 2017: 3 · 2016: 1
- FWCI
- 2.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Briefings in Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Keywords
- differential network analysis, network topology comparison, transcriptome, cancer, regulatory networks, co-expression network, microRNA, key player identification, differential expression analysis, algorithm comparison
- MeSH terms
- Algorithms, Computational Biology, Gene Expression Profiling, MicroRNAs, Gene Regulatory Networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
