Robust and sparse correlation matrix estimation for the analysis of high-dimensional genomics data
Abstract
Motivation: Microarray technology can be used to study the expression of thousands of genes across a number of different experimental conditions, usually hundreds. The underlying principle is that genes sharing similar expression patterns, across different samples, can be part of the same co-expression system, or they may share the same biological functions. Groups of genes are usually identified based on cluster analysis. Clustering methods rely on the similarity matrix between genes. A common choice to measure similarity is to compute the sample correlation matrix. Dimensionality reduction is another popular data analysis task which is also based on covariance/correlation matrix estimates. Unfortunately, covariance/correlation matrix estimation suffers from the intrinsic noise present in high-dimensional data. Sources of noise are: sampling variations, presents of outlying sample units, and the fact that in most cases the number of units is much larger than the number of genes. Results: In this paper, we propose a robust correlation matrix estimator that is regularized based on adaptive thresholding. The resulting method jointly tames the effects of the high-dimensionality, and data contamination. Computations are easy to implement and do not require hand tunings. Both simulated and real data are analyzed. A Monte Carlo experiment shows that the proposed method is capable of remarkable performances. Our correlation metric is more robust to outliers compared with the existing alternatives in two gene expression datasets. It is also shown how the regularization allows to automatically detect and filter spurious correlations. The same regularization is also extended to other less robust correlation measures. Finally, we apply the ARACNE algorithm on the SyNTreN gene expression data. Sensitivity and specificity of the reconstructed network is compared with the gold standard. We show that ARACNE performs better when it takes the proposed correlation matrix estimator as input. Availability and implementation: The R software is available at https://github.com/angy89/RobustSparseCorrelation. Contact: aserra@unisa.it or robtag@unisa.it. Supplementary information: Supplementary data are available at Bioinformatics online.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commonly used similarity measures may not be robust or efficient for measuring distance between two sets of high-dimensional sparse scRNA-seq data.
“Meanwhile, the commonly-used similarity measures (i.e.cosine similarity, Spearman's correlation, and Pearson correlation) may not be robust or efficient for measuring the distance between two sets of high-dimensional sparse scRNA-seq data 14 .”
From scBERT as a Large-scale Pretrained Deep Language Model for Cell Type Annotation of Single-cell RNA-seq Data (2021) · cites this paper as [14]
Cites
Sparse inverse covariance estimation with the graphical lasso
Friedman, J., Hastie, T., & Tibshirani, R. (2008) · Biostatistics · doi:10.1093/biostatistics/kxm045 · Cited by 6,355 · see citing works
"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207 · see citing works
A NEW MEASURE OF RANK CORRELATION
Kendall, M. G. (1938) · Biometrika · doi:10.1093/biomet/30.1-2.81 · Cited by 5,988 · see citing works
A General Framework for Weighted Gene Co-Expression Network Analysis
Zhang, B., & Horvath, S. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1128 · Cited by 5,684 · see citing works
Yang, Y., Dudoit, S., Luu, P., Lin, D., Peng, V., Ngai, J., & Speed, T. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.4.e15 · Cited by 3,722 · see citing works
A well-conditioned estimator for large-dimensional covariance matrices
Ledoit, O., & Wolf, M. (2004) · Journal of Multivariate Analysis · doi:10.1016/s0047-259x(03)00096-4 · Cited by 3,046 · see citing works
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420 · see citing works
Margolin, A. A., Nemenman, I. M., Basso, K., Wiggins, C. H., Stolovitzky, G. A., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.7916/d8g15zbp · Cited by 2,260 · see citing works
Model selection and estimation in the Gaussian graphical model
Yuan, M., & Lin, Y. (2007) · Biometrika · doi:10.1093/biomet/asm018 · Cited by 1,627 · see citing works
The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal?
Michaud, R. O. (1989) · Financial Analysts Journal · doi:10.2469/faj.v45.n1.31 · Cited by 1,384 · see citing works
Cluster analysis for gene expression data: a survey
Jiang, D., Tang, C., & Zhang, A. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.68 · Cited by 1,238 · see citing works
Covariance regularization by thresholding
Bickel, P. J., & Levina, E. (2007) · Cited by 1,118 · see citing works
A Constrained ℓ 1 Minimization Approach to Sparse Precision Matrix Estimation
Cai, T., Liu, W., & Luo, X. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10155 · Cited by 1,045 · see citing works
Computing the nearest correlation matrix--a problem from finance
Higham, N. J. (2002) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/22.3.329 · Cited by 900 · see citing works
Covariance regularization by thresholding
Bickel, P. J., & Levina, E. (2008) · The Annals of Statistics · doi:10.1214/08-aos600 · Cited by 895 · see citing works
Robust Estimates, Residuals, and Outlier Detection with Multiresponse Data
Gnanadesikan, R., & Kettenring, J. R. (1972) · Biometrics · doi:10.2307/2528963 · Cited by 826 · see citing works
2014 International Joint Conference on Neural Networks (IJCNN)
(2014) · doi:10.1109/ijcnn32435.2014 · Cited by 699 · see citing works
Sparse Permutation Invariant Covariance Estimation
Rothman, A., Bickel, P. J., Levina, E., & Zhu, J. (2007) · Cited by 686 · see citing works
Adaptive Thresholding for Sparse Covariance Matrix Estimation
Cai, T., & Liu, W. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10560 · Cited by 601 · see citing works
Sparse permutation invariant covariance estimation
Rothman, A. J., Bickel, P. J., Levina, E., & Zhu, J. (2008) · Electronic Journal of Statistics · doi:10.1214/08-ejs176 · Cited by 562 · see citing works
Generalized Thresholding of Large Covariance Matrices
Rothman, A. J., Levina, E., & Zhu, J. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.0101 · Cited by 495 · see citing works
Similarities and Differences in Genome-Wide Expression Data of Six Organisms
Bergmann, S., Ihmels, J., & Barkai, N. (2003) · PLoS Biology · doi:10.1371/journal.pbio.0020009 · Cited by 348 · see citing works
Gene expression analysis identifies global gene dosage sensitivity in cancer
Fehrmann, R. S. N., Karjalainen, J. M., Krajewska, M., Westra, H.-J., Maloney, D., Simeonov, A., Pers, T. H., Hirschhorn, J. N., Jansen, R. C., Schultes, E. A., et al. (2015) · Nature Genetics · doi:10.1038/ng.3173 · Cited by 347 · see citing works
Operator norm consistent estimation of large-dimensional sparse covariance matricesPreprint
Karoui, N. E. (2012) · Cited by 345 · see citing works
First-Order Methods for Sparse Covariance Selection
d’Aspremont, A., Banerjee, O., & Ghaoui, L. E. (2008) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/060670985 · Cited by 343 · see citing works
Getting the Noise Out of Gene Arrays
Marshall, E. (2004) · Science · doi:10.1126/science.306.5696.630 · Cited by 302 · see citing works
den Bulcke, T. V., Leemput, K. V., Naudts, B., van Remortel, P., Ma, H., Verschoren, A. H. M. J., Moor, B. D., & Kathleen Marchal, D. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-43 · Cited by 294 · see citing works
Microarrays and molecular research: noise discovery?
Ioannidis, J. P. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17878-7 · Cited by 271 · see citing works
Operator norm consistent estimation of large-dimensional sparse covariance matrices
Karoui, N. E. (2008) · The Annals of Statistics · doi:10.1214/07-aos559 · Cited by 245 · see citing works
High‐Dimensional Covariance Estimation
Pourahmadi, M. (2013) · Wiley series in probability and statistics · doi:10.1002/9781118573617 · Cited by 163 · see citing works
Condition-Number-Regularized Covariance Estimation
Won, J., Lim, J., Kim, S.-J., & Rajaratnam, B. (2012) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2012.01049.x · Cited by 159 · see citing works
CoExpNetViz: Comparative Co-Expression Networks Construction and Visualization Tool
Tzfadia, O., Diels, T., Meyer, S. D., Vandepoele, K., Aharoni, A., & de Peer, Y. V. (2016) · Frontiers in Plant Science · doi:10.3389/fpls.2015.01194 · Cited by 115 · see citing works
Maximum-likelihood estimation of the parameters of a multivariate normal distribution
Anderson, T. W., & Olkin, I. (1985) · Linear Algebra and its Applications · doi:10.1016/0024-3795(85)90049-7 · Cited by 111 · see citing works
D’haeseleer, P., Wen, X., Fuhrman, S., & Somogyi, R. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5345-8_22 · Cited by 105 · see citing works
On a Robust Correlation Coefficient
Abdullah, M. B. (1990) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2349088 · Cited by 98 · see citing works
A robust measure of correlation between two genes on a microarray
Hardin, J., Mitani, A., Hicks, L., & Koten, B. V. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-220 · Cited by 91 · see citing works
Information Processing in Cells and Tissues
Holcombe, M., & Paton, R. (1998) · Springer eBooks · doi:10.1007/978-1-4615-5345-8 · Cited by 90 · see citing works
Robust Estimation of the Correlation Coefficient: An Attempt of Survey
Shevlyakov, G. L., & Smirnov, P. O. (2010) · Austrian Journal of Statistics · doi:10.17713/ajs.v40i1&2.206 · Cited by 90 · see citing works
Kernel hierarchical gene clustering from microarray expression data
Qin, J., Lewis, D. P., & Noble, W. S. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg288 · Cited by 67 · see citing works
A study of inter-lab and inter-platform agreement of DNA microarray data
WANG, H. J., He, X., Band, M., Wilson, C., & Liu, L. (2005) · BMC Genomics · doi:10.1186/1471-2164-6-71 · Cited by 66 · see citing works
Robust estimation of precision matrices under cellwise contamination
Tarr, G., Müller, S., & Weber, N. C. (2015) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2015.02.005 · Cited by 58 · see citing works
Bickel, D. R. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg092 · Cited by 55 · see citing works
Robust High-Dimensional Precision Matrix Estimation
Öllerer, V., & Croux, C. (2015) · Springer eBooks · doi:10.1007/978-3-319-22404-6_19 · Cited by 44 · see citing works
Improved direct LDA and its application to DNA microarray gene expression data
Paliwal, K. K., & Sharma, A. K. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.08.003 · Cited by 39 · see citing works
Modern Nonparametric, Robust and Multivariate Methods
Nordhausen, K., & Taskinen, S. (2015) · Springer eBooks · doi:10.1007/978-3-319-22404-6 · Cited by 19 · see citing works
Covariance selection by thresholding the sample correlation matrix
Jiang, B. (2013) · Statistics & Probability Letters · doi:10.1016/j.spl.2013.07.008 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., & Yao, J. (2021) · doi:10.1101/2021.12.05.471261 · Cited by 617 · see citing works
Linear and nonlinear correlation estimators unveil undescribed taxa interactions in microbiome data
Huang, L., Eggesbø, M. Å., & Peddada, S. (2022) · Nature Communications · doi:10.1038/s41467-022-32243-x · Cited by 159 · see citing works
Transcriptomics in Toxicogenomics, Part III: Data Modelling for Risk Assessment
Serra, A., Fratello, M., Cattelani, L., Liampa, I., Melagraki, G., Kohonen, P. J., Nymark, P. E. H., Federico, A., Kinaret, P., Jagiełło, K., et al. (2020) · Nanomaterials · doi:10.3390/nano10040708 · Cited by 72 · see citing works
Self-citationRobust clustering of noisy high-dimensional gene expression data for patients subtyping
Coretto, P., Serra, A., & Tagliaferri, R. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty502 · Cited by 31 · see citing works
Self-citationStatistical genomics in rare cancer
Aghababazadeh, F. A., Mo, Q., & Fridley, B. L. (2019) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2019.08.021 · Cited by 27 · see citing works
Khodayar, M., Saffari, M., Williams, M., & Jalali, S. M. J. (2022) · Energy · doi:10.1016/j.energy.2022.124143 · Cited by 23 · see citing works
Revealing gene function with statistical inference at single-cell resolution
Trapnell, C. (2024) · Nature Reviews Genetics · doi:10.1038/s41576-024-00750-w · Cited by 19 · see citing works
A set-level joint sparse representation for image set classification
Zheng, P., Zhao, Z., Gao, J., & Wu, X. (2018) · Information Sciences · doi:10.1016/j.ins.2018.02.062 · Cited by 19 · see citing works
A topological approach for cancer subtyping from gene expression data
Rafique, O., & Mir, A. H. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103357 · Cited by 16 · see citing works
Low-Rank Linear Embedding for Robust Clustering
Zhou, J., Pedrycz, W., Wan, J., Gao, C., Lai, Z., & Yue, X. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3144294 · Cited by 15 · see citing works
Network Analysis of Microarray Data
Pavel, A., Serra, A., Cattelani, L., Federico, A., & Greco, D. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1839-4_11 · Cited by 11 · see citing works
Self-citationRather, A. A., & Chachoo, M. A. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106640 · Cited by 17 · see citing works
Rafique, O., & Mir, A. H. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103620 · Cited by 11 · see citing works
XGRN: Reconstruction of Biological Networks Based on Boosted Trees Regression
Dimitrakopoulos, G. Ν. (2021) · Computation · doi:10.3390/computation9040048 · Cited by 7 · see citing works
UMAP guided topological analysis of transcriptomic data for cancer subtyping
Rather, A. A., & Chachoo, M. A. (2022) · International Journal of Information Technology · doi:10.1007/s41870-022-01048-y · Cited by 3 · see citing works
Yu, Z., Ye, Y., & Pan, J. (2025) · Journal of Advanced Research · doi:10.1016/j.jare.2025.10.056 · Cited by 2 · see citing works
Correlations for untargeted GC × GC-HRTOF-MS metabolomics of colorectal cancer
Giovanni, N. D., Meuwis, M., Louis, E. J., & Focant, J. (2023) · Metabolomics · doi:10.1007/s11306-023-02047-1 · Cited by 2 · see citing works
Wei, S., Lu, Y., Wang, P., Li, Q., Shuai, J., Zhao, Q., Lin, H., & Peng, Y. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-06263-2 · Cited by 2 · see citing works
RSC-based differential model with correlation removal for improving multi-omics clustering
Lu, Z., Chen, X., Yang, J., & Ding, Y. (2022) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2022.111328 · Cited by 2 · see citing works
A dual active-set proximal Newton algorithm for sparse approximation of correlation matrices
Liu, X., Shen, C., & Wang, L. (2022) · Optimization methods & software · doi:10.1080/10556788.2021.1998491 · Cited by 1 · see citing works
KNeMAP: a network mapping approach for knowledge-driven comparison of transcriptomic profiles
Pavel, A., del Giudice, G., Fratello, M., Ghemtio, L., Lieto, A. D., Yli‐Kauhaluoma, J. T., Xhaard, H., Federico, A., Serra, A., & Greco, D. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad341 · Cited by 1 · see citing works
Self-citationA Tripartite Machine Learning Approach for Accurate Prognosis of COVID-19 Patient Survival
Aziz, F. (2023) · Journal Medical Informatics Technology · doi:10.37034/medinftech.v1i3.13 · Cited by 1 · see citing works
Inferring Independent Sets of Gaussian Variables after Thresholding Correlations
Saha, A., Witten, D., & Bien, J. (2024) · Journal of the American Statistical Association · doi:10.1080/01621459.2024.2337158 · Cited by 2 · see citing works
Molecular Subtyping of Lung Cancer Patients with Consensus Clustering
Rather, A., & Chachoo, M. A. (2022) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom54597.2022.9763214
RSC: Robust and Sparse Correlation Matrix
Coraggio, L., Coretto, P., & Serra, A. (2020) · doi:10.32614/cran.package.rsc
Self-citationWiyono, S., Adji, T. B., Nugroho, H. A., & Wibirama, S. (2026) · Information Sciences · doi:10.1016/j.ins.2026.123502
- Authors
- Serra, A., Coretto, P., Fratello, M., & Tagliaferri, R.
- Affiliations
- University of Salerno (IT)
- University of Campania "Luigi Vanvitelli" (IT)
- Corresponding author
- Serra, Angela
- Tagliaferri, Roberto
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-10-12
- Volume
- 34
- Issue
- 4
- Pages
- 625-634
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2763420753
- Cited by
- 25
- References
- 46
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 4 · 2022: 8 · 2021: 2 · 2020: 2 · 2019: 2 · 2018: 2
- FWCI
- 0.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- high-dimensional data, gene expression microarrays, adaptive thresholding, regularization, outlier robustness, gene co-expression network, ARACNE, network inference, dimensionality reduction
- MeSH terms
- Algorithms, Humans, Neoplasms, Sensitivity and Specificity, Software, Cluster Analysis, Sequence Analysis, RNA, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
