Biclustering Methods: Biological Relevance and Application in Gene Expression Analysis
Open accessAbstract
DNA microarray technologies are used extensively to profile the expression levels of thousands of genes under various conditions, yielding extremely large data-matrices. Thus, analyzing this information and extracting biologically relevant knowledge becomes a considerable challenge. A classical approach for tackling this challenge is to use clustering (also known as one-way clustering) methods where genes (or respectively samples) are grouped together based on the similarity of their expression profiles across the set of all samples (or respectively genes). An alternative approach is to develop biclustering methods to identify local patterns in the data. These methods extract subgroups of genes that are co-expressed across only a subset of samples and may feature important biological or medical implications. In this study we evaluate 13 biclustering and 2 clustering (k-means and hierarchical) methods. We use several approaches to compare their performance on two real gene expression data sets. For this purpose we apply four evaluation measures in our analysis: (1) we examine how well the considered (bi)clustering methods differentiate various sample types; (2) we evaluate how well the groups of genes discovered by the (bi)clustering methods are annotated with similar Gene Ontology categories; (3) we evaluate the capability of the methods to differentiate genes that are known to be specific to the particular sample types we study and (4) we compare the running time of the algorithms. In the end, we conclude that as long as the samples are well defined and annotated, the contamination of the samples is limited, and the samples are well replicated, biclustering methods such as Plaid and SAMBA are useful for discovering relevant subsets of genes and samples.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Different biclustering approaches can have varying results in the same data set.
“Methods should be selected carefully because different biclustering approaches can have varying results in the same data set [142].”
From Gene co-expression analysis for functional classification and gene–disease predictions (2017) · cites this paper as [142]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206 · see citing works
Algorithm AS 136: A K-Means Clustering Algorithm
Hartigan, J. A., & Wong, M. A. (1979) · Applied Statistics · doi:10.2307/2346830 · Cited by 14,523 · see citing works
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 12,617 · see citing works
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 12,153 · see citing works
Sørlie, T., Perou, C., Tibshirani, R., Aas, T., Geisler, S., Johnsen, H., Hastie, T., Eisen, M., van de Rijn, M., Jeffrey, S., et al. (2001) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.191367098 · Cited by 11,695 · see citing works
A Statistical Method for Evaluating Systematic Relationships
Sokal, R. R., & Michener, C. D. (1958) · Biodiversity Heritage Library (Smithsonian Institution) · doi:10.5281/zenodo.16435756 · Cited by 4,432 · see citing works
On the Interpretation of χ 2 from Contingency Tables, and the Calculation of P
Fisher, R. A. (1922) · Journal Of The Royal Statistical Society · doi:10.2307/2340521 · Cited by 2,975 · see citing works
Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment
Martín, M. A., Westfall, P. H., & Young, S. (1993) · Biometrics · doi:10.2307/2533464 · Cited by 2,308 · see citing works
Biclustering of expression data.
Cheng, Y., & Church, G. M. (2000) · PubMed · Cited by 2,098 · see citing works
Biclustering algorithms for biological data analysis: a survey
Madeira, S. C., & Oliveira, A. L. (2004) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2004.2 · Cited by 2,049 · see citing works
Sørensen, T. B., Sorensen, T., Biering-Sørensen, T., Ti, S., & Sorensen, J. (1948) · Medical Entomology and Zoology · Cited by 2,038 · see citing works
Novel Molecular Subtypes of Serous and Endometrioid Ovarian Cancer Linked to Clinical Outcome
Tothill, R. W., Tinker, A. V., George, J., Brown, R. E., Fox, S. B., Lade, S., Johnson, D., Trivett, M. K., Etemadmoghadam, D., Locandro, B., et al. (2008) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-08-0196 · Cited by 1,472 · see citing works
Direct Clustering of a Data Matrix
Hartigan, J. (1972) · Journal of the American Statistical Association · doi:10.1080/01621459.1972.10481214 · Cited by 1,255 · see citing works
A systematic comparison and evaluation of biclustering methods for gene expression data
Prelić, A., Stefan Bleuler, A., Philip Zimmermann, B., Wille, A., Bühlmann, P., Gruissem, W., Hennig, L., Thiele, L., & Zitzler, E. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl060 · Cited by 916 · see citing works
Discovering statistically significant biclusters in gene expression data
Tanay, A., Sharan, R., & Ron Shamir, A. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.suppl_1.s136 · Cited by 857 · see citing works
Coupled two-way clustering analysis of gene microarray data
Getz, G. A., Levine, E., & Domany, E. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.210134797 · Cited by 822 · see citing works
Spectral Biclustering of Microarray Data: Coclustering Genes and ConditionsPreprint
Kluger, Y., Basri, R., Chang, J. T., & Gerstein, M. (2003) · Genome Research · doi:10.1101/gr.648603 · Cited by 726 · see citing works
Revealing modular organization in the yeast transcriptional network
Ihmels, J., Friedlander, G., Bergmann, S., Sarig, O., Ziv, Y., & Barkai, N. (2002) · Nature Genetics · doi:10.1038/ng941 · Cited by 690 · see citing works
Discovering Local Structure in Gene Expression Data: The Order-Preserving Submatrix Problem
Ben‐Dor, A., Chor, B., Karp, R. M., & Zohar Yakhini, B. (2003) · Journal of Computational Biology · doi:10.1089/10665270360688075 · Cited by 546 · see citing works
Clustering by pattern similarity in large data sets
Wang, H., Wang, W. W., Yang, J., & Yu, P. S. (2002) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/564691.564737 · Cited by 497 · see citing works
Plaid Models for Gene Expression Data
Lazzeroni, L. C., & Owen, A. (2000) · Cited by 395 · see citing works
TiGER: A database for tissue-specific gene expression and regulation
Liu, X., Yu, X., Zack, D. J., Zhu, H., & Qian, J. X. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-271 · Cited by 391 · see citing works
EXTRACTING CONSERVED GENE EXPRESSION MOTIFS FROM GENE EXPRESSION DATA
Murali, T. M., & Kasif, S. (2002) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789812776303_0008 · Cited by 385 · see citing works
EXPANDER – an integrative program suite for microarray data analysis
Ron Shamir, A., Maron‐Katz, A., Tanay, A., Linhart, C., Steinfeld, I., Sharan, R., Shiloh, Y., & Elkon, R. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-232 · Cited by 340 · see citing works
Defining transcription modules using large-scale gene expression data
Ihmels, J., Bergmann, S., & Barkai, N. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth166 · Cited by 338 · see citing works
FABIA: factor analysis for bicluster acquisition
Hochreiter, S., Bodenhofer, U., Heusel, M., Mayr, A., Mitterecker, A., Kasim, A., Khamiakova, T., Sanden, S. V., Lin, D. Y., Talloen, W., et al. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq227 · Cited by 297 · see citing works
Kilpinen, S. K., Autio, R., Ojala, K., Iljin, K., Bucher, E., Sara, H., Pisto, T., Saarela, M., Skotheim, R. I., Björkman, M. L., et al. (2008) · Genome biology · doi:10.1186/gb-2008-9-9-r139 · Cited by 283 · see citing works
Self-citationNext generation software for functional trend analysis
Berriz, G. F., Beaver, J. E., Cenik, C., Tasan, M., & Roth, F. P. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp498 · Cited by 277 · see citing works
BicAT: a biclustering analysis toolbox
Barkow‐Oesterreicher, S., Stefan Bleuler, A., Prelić, A., Philip Zimmermann, B., & Zitzler, E. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl099 · Cited by 273 · see citing works
Busygin, S., Prokopyev, O. A., & Pardalos, P. M. (2007) · Computers & Operations Research · doi:10.1016/j.cor.2007.01.005 · Cited by 270 · see citing works
QUBIC: a qualitative biclustering algorithm for analyses of gene expression data
Li, G., Ma, Q., Tang, H., Paterson, A. H., & Xu, Y. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp491 · Cited by 269 · see citing works
Reiss, D. J., Baliga, N. S., & Bonneau, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-280 · Cited by 253 · see citing works
Handbook of computational molecular biology
(2006) · Choice Reviews Online · doi:10.5860/choice.43-6513 · Cited by 236 · see citing works
A comparative analysis of biclustering algorithms for gene expression data
Eren, K., Deveci, M., Kucuktunc, O., & Çatalyürek, Ü. V. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbs032 · Cited by 229 · see citing works
Biclustering microarray data by Gibbs sampling
Sheng, Q., Moreau, Y., & Moor, B. D. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg1078 · Cited by 222 · see citing works
A roadmap of clustering algorithms: finding a match for a biomedical application
Andreopoulos, B., An, A., Wang, X., & Schroeder, M. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn058 · Cited by 221 · see citing works
OP-cluster: clustering by tendency in high dimensional space
Liu, J., & Wang, W. W. (2004) · Proceedings · doi:10.1109/icdm.2003.1250919 · Cited by 197 · see citing works
Analysis of gene expression microarrays for phenotype classification.
Califano, A., Stolovitzky, G. A., & Tu, Y. (2000) · PubMed · Cited by 188 · see citing works
Analysis and visualization of gene expression data using Self-Organizing Maps
Nikkilä, J., Törönen, P., Samuel Kaski, A., Venna, J., Ċastrén, E., & Wong, G. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00070-9 · Cited by 179 · see citing works
CancerGenes: a gene selection resource for cancer genome projects
Higgins, M. E., Claremont, M., Major, J. E., Sander, C., & Lash, A. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl811 · Cited by 179 · see citing works
Improved biclustering of microarray data demonstrated through systematic performance tests
Turner, H., Bailey, T., & Krzanowski, W. J. (2004) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2004.02.003 · Cited by 166 · see citing works
Interrelated two-way clustering: an unsupervised approach for gene expression data analysis
Tang, C., Zhang, L., Zhang, A., et al. (2001) · Proceedings-- IEEE International Symposium on Bioinformatics and Bioengineering · doi:10.1109/bibe.2001.974410 · Cited by 161 · see citing works
Bayesian biclustering of gene expression data
Gu, J., & Liu, J. S. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-s1-s4 · Cited by 158 · see citing works
Computing the maximum similarity bi-clusters of gene expression data
Liu, X., & Wang, L. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl560 · Cited by 145 · see citing works
AN IMPROVED BICLUSTERING METHOD FOR ANALYZING GENE EXPRESSION PROFILES
Yang, J., Wang, H., Wang, W., et al. (2005) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213005002387 · Cited by 91 · see citing works
A biclustering algorithm based on a Bicluster Enumeration Tree: application to DNA microarray data
Ayadi, W., Elloumi, M., & Hao, J. (2009) · BioData Mining · doi:10.1186/1756-0381-2-9 · Cited by 79 · see citing works
A Toolbox for Bicluster Analysis in R
Kaiser, S., & Leisch, F. (2008) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.3293 · Cited by 65 · see citing works
FABIA: factor analysis for bicluster acquisition
Hochreiter, S., Bodenhofer, U., Heusel, M., et al. (2010) · F1000Research · doi:10.7490/f1000research.256.1 · Cited by 64 · see citing works
Teng, L., & Chan, L. (2007) · Journal of Signal Processing Systems · doi:10.1007/s11265-007-0121-2 · Cited by 62 · see citing works
Yoon, S., Nardini, C., Benini, L., et al. (2005) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2005.55 · Cited by 61 · see citing works
Gene Ontology friendly biclustering of expression profiles.
Liu, J., Wang, W., & Yang, J. (2004) · PubMed · Cited by 54 · see citing works
Autio, R., Kilpinen, S. K., Saarela, M., et al. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-s1-s24 · Cited by 48 · see citing works
Self-citationGEMS: a web server for biclustering analysis of expression data
Wu, C., & Kasif, S. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki469 · Cited by 42 · see citing works
Methods to Bicluster Validation and Comparison in Microarray Data
Santamaría, R., Quintales, L. A. M., & Therón, R. (2007) · Springer eBooks · doi:10.1007/978-3-540-77226-2_78 · Cited by 39 · see citing works
Bayesian biclustering with the plaid model
Caldas, J., & Kaski, S. (2008) · Machine learning for signal processing ... · doi:10.1109/mlsp.2008.4685495 · Cited by 35 · see citing works
Gene expression module discovery using gibbs sampling.
Wu, C., Fu, Y., Murali, T. M., et al. (2004) · PubMed · Cited by 32 · see citing works
Hierarchical Generative Biclustering for MicroRNA Expression Analysis
Caldas, J., & Samuel Kaski, A. (2011) · Journal of Computational Biology · doi:10.1089/cmb.2010.0256 · Cited by 29 · see citing works
Li, L., Guo, Y., Wu, W., Shi, Y., Cheng, J. F., & Tao, S. (2012) · BioData Mining · doi:10.1186/1756-0381-5-8 · Cited by 29 · see citing works
Wang, Y. K., Print, C. G., & Crampin, E. J. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-102 · Cited by 28 · see citing works
Exhaustive Search Method of Gene Expression Modules and Its Application to Human Tissue Data
Okada, Y., Okubo, K., Horton, P., & Fujibuchi, W. (2007) · Cited by 15 · see citing works
A GENERAL FRAMEWORK FOR BICLUSTERING GENE EXPRESSION DATA
Li, H., CHEN, X., Zhang, K., & Jiang, T. (2006) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s021972000600217x · Cited by 12 · see citing works
Hierarchical Generative Biclustering for MicroRNA Expression Analysis
Caldas, J., & Samuel Kaski, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12683-3_5 · Cited by 8 · see citing works
An Efficient Voting Algorithm for Finding Additive Biclusters with Random Background
Xiao, J., Wang, L., Liu, X., & Jiang, T. (2008) · Journal of Computational Biology · doi:10.1089/cmb.2007.0219 · Cited by 5 · see citing works
den Bulcke, T. V. (2009) · Cited by 3 · see citing works
Clustering Algorithms for Gene Expression Analysis
Baldi, P., Hatfield, G. W., & Fu, L. (2005) · Chapman & Hall/CRC computer and information science series · doi:10.1201/9781420036275.ch25 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [22]22. den Bulcke TV (2009) Robust algorithms for inferring regulatory networks based on gene expression measurements and biological prior information. Ph.D. thesis, Katholieke Universiteit Leuven.
- [55]55. Busygin S, Jacobsen G, Kramer E (2002) Double conjugated clustering applied to leukemia microarray data. In: Proc. Second SIAM Int'l Conf. Data Mining, Workshop on clustering high dimensional data.
Cited by
Sorted by citations
Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 1,086 · see citing works
Analysis of miRNA expression profiles in breast cancer using biclustering
Fiannaca, A., La Rosa, M., La Paglia, L., Rizzo, R., & Urso, A. (2015) · BMC Bioinformatics · doi:10.1186/1471-2105-16-s4-s7 · Cited by 45 · see citing works
- Authors
- Oghabian, A., Kilpinen, S., Hautaniemi, S., & Czeizler, E.
- Affiliations
- University of Helsinki (FI)
- Aalto University (FI)
- Corresponding author
- Czeizler, Elena
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-03-20
- Volume
- 9
- Issue
- 3
- Pages
- e90801
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3961251
- OpenAlex
- W1993445335
- Cited by
- 88
- References
- 65
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 1 · 2023: 3 · 2022: 2 · 2021: 6 · 2020: 9 · 2019: 13 · 2018: 14 · 2017: 11 · 2016: 12 · 2015: 13
- FWCI
- 4.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Aaltodoc (Aalto University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Työväentutkimus Vuosikirja (submittedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Clustering Algorithms Research
- Keywords
- biclustering, gene expression analysis, microarray data, clustering algorithm comparison, K-means clustering, hierarchical clustering, gene ontology annotation, plaid, samba, algorithm running time, sample contamination
- MeSH terms
- Breast Neoplasms, Female, Gene Expression Regulation, Humans, Cluster Analysis, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
