The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
An overview of the estimation of large covariance and precision matrices
Abstract
The estimation of large covariance and precision matrices is fundamental in modern multivariate analysis. However, problems arise from the statistical analysis of large panel economic and financial data. The covariance matrix reveals marginal correlations between variables, while the precision matrix encodes conditional correlations between pairs of variables given the remaining variables. In this paper, we provide a selective review of several recent developments on the estimation of large covariance and precision matrices. We focus on two general approaches: a rank‐based method and a factor‐model‐based method. Theories and applications of both approaches are presented. These methods are expected to be widely applicable to the analysis of economic and financial data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cross‐Section of Expected Stock Returns
FAMA, E. F., & FRENCH, K. R. (1992) · The Journal of Finance · doi:10.1111/j.1540-6261.1992.tb04398.x · Cited by 17,261
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
Self-citationThe adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470
Sparse inverse covariance estimation with the graphical lasso
Friedman, J., Hastie, T., & Tibshirani, R. (2008) · Biostatistics · doi:10.1093/biostatistics/kxm045 · Cited by 6,355
The Econometrics of Financial Markets
Campbell, J. Y., Lo, A. W., & MacKinlay, A. C. (2012) · Princeton University Press eBooks · doi:10.2307/j.ctt7skm5 · Cited by 5,875
Introduction to Numerical Analysis.
Froberg, C.-E., Stoer, J., & Bulirsch, R. (1981) · Mathematics of Computation · doi:10.2307/2007454 · Cited by 5,463
Exact Matrix Completion via Convex Optimization
Candès, E. J., & Recht, B. (2009) · Foundations of Computational Mathematics · doi:10.1007/s10208-009-9045-5 · Cited by 4,877
Determining the Number of Factors in Approximate Factor Models
Bai, J., & Ng, S. (2002) · Econometrica · doi:10.1111/1468-0262.00273 · Cited by 4,099
Nearly unbiased variable selection under minimax concave penalty
Zhang, C.-H. (2010) · The Annals of Statistics · doi:10.1214/09-aos729 · Cited by 3,835
A well-conditioned estimator for large-dimensional covariance matrices
Ledoit, O., & Wolf, M. (2004) · Journal of Multivariate Analysis · doi:10.1016/s0047-259x(03)00096-4 · Cited by 3,046
Forecasting Using Principal Components From a Large Number of Predictors
Stock, J. H., & Watson, M. W. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618960 · Cited by 2,985
High dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · Cited by 2,539
High-dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000281 · Cited by 2,521
Simultaneous analysis of Lasso and Dantzig selector
Bickel, P. J., Ritov, Y., & Tsybakov, A. B. (2009) · The Annals of Statistics · doi:10.1214/08-aos620 · Cited by 2,396
Introduction to the non-asymptotic analysis of random matrices
Vershynin, R. (2012) · Compressed Sensing · doi:10.1017/cbo9780511794308.006 · Cited by 2,150
On the distribution of the largest eigenvalue in principal components analysis
Johnstone, I. M. (2001) · The Annals of Statistics · doi:10.1214/aos/1009210544 · Cited by 1,949
Wavelet Shrinkage: Asymptopia?
Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., & Picard, D. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02032.x · Cited by 1,749
The Dantzig selector: Statistical estimation when p is much larger than n
Candes, E., & Tao, T. (2007) · The Annals of Statistics · doi:10.1214/009053606000001523 · Cited by 1,747
The Generalized Dynamic-Factor Model: Identification and Estimation
Forni, M., Hallin, M., Lippi, M., & Reichlin, L. (2000) · Review of Economics and Statistics · doi:10.1162/003465300559037 · Cited by 1,709
Inferential Theory for Factor Models of Large Dimensions
Bai, J. (2003) · Econometrica · doi:10.1111/1468-0262.00392 · Cited by 1,678
Model selection and estimation in the Gaussian graphical model
Yuan, M., & Lin, Y. (2007) · Biometrika · doi:10.1093/biomet/asm018 · Cited by 1,627
Ledoit, O., & Wolf, M. (2003) · Journal of Empirical Finance · doi:10.1016/s0927-5398(03)00007-0 · Cited by 1,627
Challenges of Big Data analysis
Fan, J., Han, F., & Liu, H. (2014) · National Science Review · doi:10.1093/nsr/nwt032 · Cited by 1,482
Self-citationFactor Analysis as a Statistical Method.
Basilevsky, A., Lawley, D. N., & Maxwell, A. E. (1972) · The Statistician · doi:10.2307/2986693 · Cited by 1,471
Symmetric Multivariate and Related Distributions
Fang, K.-T., Kotz, S., & Ng, K. W. (1990) · Springer eBooks · doi:10.1007/978-1-4899-2937-2 · Cited by 1,254
Banerjee, O., Ghaoui, L. E., & d’Aspremont, A. (2007) · arXiv (Cornell University) · doi:10.1145/1390681.1390696 · Cited by 1,233
Wainwright, M. J. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2016018 · Cited by 1,198
Introduction to Numerical Analysis
Stoer, J., & Bulirsch, R. Z. (1993) · Texts in applied mathematics · doi:10.1007/978-1-4757-2272-7 · Cited by 1,167
Covariance regularization by thresholding
Bickel, P. J., & Levina, E. (2007) · Cited by 1,118
Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
Chamberlain, G., & Rothschild, M. (1983) · Econometrica · doi:10.2307/1912275 · Cited by 1,087
Nonconcave penalized likelihood with a diverging number of parameters
Fan, J., & Peng, H. (2004) · The Annals of Statistics · doi:10.1214/009053604000000256 · Cited by 1,002
Self-citationOn Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
Chamberlain, G., & Rothschild, M. (1982) · National Bureau of Economic Research · doi:10.3386/w0996 · Cited by 975
A Constrained ℓ1Minimization Approach to Sparse Precision Matrix Estimation
Cai, T., Liu, W., & Luo, X. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10155 · Cited by 973
Eigenvalue Ratio Test for the Number of Factors
(2013) · Econometrica · doi:10.3982/ecta8968 · Cited by 936
Linear Programming: Foundations and Extensions
Vanderbei, R. J. (1998) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600987 · Cited by 921
Large Covariance Estimation by Thresholding Principal Orthogonal Complements
Fan, J., Liao, Y., & Mincheva, M. (2013) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12016 · Cited by 909
Self-citationCovariance regularization by thresholding
Bickel, P. J., & Levina, E. (2008) · The Annals of Statistics · doi:10.1214/08-aos600 · Cited by 893
The econometrics of financial markets
Pagan, A. (1996) · Journal of Empirical Finance · doi:10.1016/0927-5398(95)00020-8 · Cited by 873
Are more data always better for factor analysis?
Boivin, J., & Ng, S. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2005.01.027 · Cited by 811
Optimization Transfer Using Surrogate Objective Functions
Lange, K., Hunter, D. R., & Yang, I. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474858 · Cited by 808
Sparse Permutation Invariant Covariance Estimation
Rothman, A., Bickel, P. J., Levina, E., & Zhu, J. (2007) · Cited by 686
High dimensional covariance matrix estimation using a factor model
Fan, J., Fan, Y., & Lv, J. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2008.09.017 · Cited by 685
Self-citationNuclear-norm penalization and optimal rates for noisy low-rank matrix completion
Koltchinskii, V., Lounici, K., & Tsybakov, A. B. (2011) · The Annals of Statistics · doi:10.1214/11-aos894 · Cited by 598
Adaptive Thresholding for Sparse Covariance Matrix Estimation
Cai, T., & Liu, W. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10560 · Cited by 590
ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL
Paul, D. (2007) · Cited by 577
Sparse permutation invariant covariance estimation
Rothman, A. J., Bickel, P. J., Levina, E., & Zhu, J. (2008) · Electronic Journal of Statistics · doi:10.1214/08-ejs176 · Cited by 560
Determining the Number of Factors in the General Dynamic Factor Model
Hallin, M., & Liška, R. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001275 · Cited by 547
Estimation of (near) low-rank matrices with noise and high-dimensional scaling
Negahban, S., & Wainwright, M. J. (2011) · The Annals of Statistics · doi:10.1214/10-aos850 · Cited by 537
A Quasi–Maximum Likelihood Approach for Large, Approximate Dynamic Factor Models
Doz, C., Giannone, D., & Reichlin, L. (2012) · Review of Economics and Statistics · doi:10.1162/rest_a_00225 · Cited by 530
Factor modeling for high-dimensional time series: Inference for the number of factors
Lam, C., & Yao, Q. (2012) · The Annals of Statistics · doi:10.1214/12-aos970 · Cited by 514
High-dimensional semiparametric Gaussian copula graphical models
Liu, H., Han, F., Yuan, M., Lafferty, J., & Wasserman, L. (2012) · The Annals of Statistics · doi:10.1214/12-aos1037 · Cited by 507
Self-citationRegularization of Wavelet Approximations
Antoniadis, A., & Fan, J. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753208942 · Cited by 498
Self-citationGeneralized Thresholding of Large Covariance Matrices
Rothman, A. J., Levina, E., & Zhu, J. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.0101 · Cited by 489
Sparsistency and rates of convergence in large covariance matrix estimation
Lam, C., & Fan, J. (2009) · The Annals of Statistics · doi:10.1214/09-aos720 · Cited by 418
Self-citationNetwork exploration via the adaptive LASSO and SCAD penalties
Fan, J., Feng, Y., & Wu, Y. (2009) · The Annals of Applied Statistics · doi:10.1214/08-aoas215 · Cited by 397
Self-citationSparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana
Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelić, A., von Rohr, P., Thiele, L., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-11-r92 · Cited by 380
On the Class of Elliptical Distributions and their Applications to the Theory of Portfolio Choice
OWEN, J., & RABINOVITCH, R. (1983) · The Journal of Finance · doi:10.1111/j.1540-6261.1983.tb02499.x · Cited by 372
High-dimensional covariance matrix estimation in approximate factor models
Fan, J., Liao, Y., & Mincheva, M. (2011) · The Annals of Statistics · doi:10.1214/11-aos944 · Cited by 358
Self-citationHigh Dimensional Inverse Covariance Matrix Estimation via Linear Programming
Yuan, M. (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859930 · Cited by 354
Vast Portfolio Selection With Gross-Exposure Constraints
Fan, J., Zhang, J., & Yu, K. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.682825 · Cited by 336
Self-citationOnatski, A. (2012) · Journal of Econometrics · doi:10.1016/j.jeconom.2012.01.034 · Cited by 314
Sparse principal component analysis and iterative thresholding
Ma, Z. (2013) · The Annals of Statistics · doi:10.1214/13-aos1097 · Cited by 313
Statistical analysis of factor models of high dimension
Bai, J., & Li, K. (2012) · The Annals of Statistics · doi:10.1214/11-aos966 · Cited by 310
PCA consistency in high dimension, low sample size context
Jung, S., & Marron, J. S. (2009) · The Annals of Statistics · doi:10.1214/09-aos709 · Cited by 303
Nonparametric estimation of large covariance matrices of longitudinal data
Wu, W. B. (2003) · Biometrika · doi:10.1093/biomet/90.4.831 · Cited by 300
Challenging the empirical mean and empirical variance: A deviation study
Catoni, O. (2012) · Annales de l'Institut Henri Poincaré, Probabilités et Statistiques · doi:10.1214/11-aihp454 · Cited by 298
Sparse PCA: Optimal rates and adaptive estimation
Cai, T. T., Ma, Z., & Wu, Y. (2013) · The Annals of Statistics · doi:10.1214/13-aos1178 · Cited by 283
Optimal Rates of Convergence for Sparse Covariance Matrix Estimation
Cai, T., & Zhou, H. H. (2012) · ScholarlyCommons (University of Pennsylvania) · Cited by 243
Understanding and Comparing Factor-Based Forecasts ∗
Jean Boivin, A., Ng, S., Giannone, W. T. D., & Forni, M. (2014) · Research Publication Repository of King Fahd University of Petroleum and Minerals (King Fahd University of Petroleum and Minerals) · Cited by 242
Optimal rates of convergence for sparse covariance matrix estimation
Cai, T., & Zhou, H. H. (2012) · The Annals of Statistics · doi:10.1214/12-aos998 · Cited by 235
Wavelets in statistics: A review
Antoniadis, A. (1997) · Journal of the Italian Statistical Society · doi:10.1007/bf03178905 · Cited by 229
Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
Fan, J., Li, Q., & Wang, Y. (2017) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12166 · Cited by 214
Self-citationEfficient Semiparametric Estimation of the Fama-French Model and Extensions
(2012) · Econometrica · doi:10.3982/ecta7432 · Cited by 185
Regularized rank-based estimation of high-dimensional nonparanormal graphical models
Xue, L., & Zou, H. (2012) · The Annals of Statistics · doi:10.1214/12-aos1041 · Cited by 180
Projected principal component analysis in factor models
Fan, J., Liao, Y., & Wang, W. (2015) · The Annals of Statistics · doi:10.1214/15-aos1364 · Cited by 162
Self-citationHigh‐Dimensional Covariance Estimation
Pourahmadi, M. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118573617 · Cited by 162
High-dimensional instrumental variables regression and confidence setsPreprint
Gautier, É., & Rose, C. D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.2454 · Cited by 157
The high-dimension, low-sample-size geometric representation holds under mild conditions
Ahn, J. Y., Marron, J. S., Muller, K. M., & Chi, Y. (2007) · Biometrika · doi:10.1093/biomet/asm050 · Cited by 148
Projected Principal Component Analysis in Factor ModelsPreprint
Fan, J., Liao, Y., & Wang, W. (2016) · Cited by 138
Self-citationVast volatility matrix estimation for high-frequency financial data
Wang, Y., & Zou, J. (2010) · The Annals of Statistics · doi:10.1214/09-aos730 · Cited by 129
Karoui, N. E. (2010) · The Annals of Statistics · doi:10.1214/10-aos795 · Cited by 108
Large covariance estimation through elliptical factor models
Fan, J., Liu, H., & Wang, W. (2018) · The Annals of Statistics · doi:10.1214/17-aos1588 · Cited by 107
Self-citationConsistency of sparse PCA in High Dimension, Low Sample Size contexts
Shen, D., Shen, H., & Marron, J. S. (2012) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2012.10.007 · Cited by 101
The rotation of eigenvectors by a perturbation
Davis, C. (1963) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(63)90001-5 · Cited by 98
Semiparametric estimation of a characteristic-based factor model of common stock returns
Connor, G., & Linton, O. (2006) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2006.10.001 · Cited by 88
Sparse Covariance Matrix Estimation With Eigenvalue Constraints
Liu, H., Wang, L.-E., & Zhao, T. (2013) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2013.782818 · Cited by 82
Self-citationStatistical analysis of big data on pharmacogenomics
B, J. F. A., & Liu, H. (2013) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2013.04.008 · Cited by 47
Self-citationSurprising Asymptotic Conical Structure in Critical Sample Eigen-DirectionsPreprint
Shen, D., Shen, H., Hongtu Zhu, B., & Marron, J. S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.6171 · Cited by 29
Calibrated Precision Matrix Estimation for High-Dimensional Elliptical Distributions
Zhao, T., & Liu, H. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2360980 · Cited by 29
Self-citationPrincipal Component Analysis of High Frequency Data
Aı̈t-Sahalia, Y., & Xiu, D. (2015) · National Bureau of Economic Research · doi:10.3386/w21584 · Cited by 24
Multivariate Analysis of Nonparametric Estimates of Large Correlation MatricesPreprint
Mitra, R., & Zhang, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.6195 · Cited by 23
Han, F., & Liu, H. (2013) · arXiv (Cornell University) · Cited by 21
Self-citation
Cited by
A comprehensive survey on regularization strategies in machine learning
Tian, Y., & Zhang, Y. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.005 · Cited by 298
Li, Y., Hu, J., Zhang, C., Yu, D.-J., & Zhang, Y. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz291 · Cited by 158
rrQNet : Protein contact map quality estimation by deep evolutionary reconciliation
(2022) · doi:10.1002/prot.26394
Metadata
- Authors
- Fan, J., Liao, Y., & Liu, H.
- Venue
- The Econometrics Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-02-01
- Volume
- 19
- Issue
- 1
- Pages
- C1-C32
- ISSN
- 1368-4221 (ISSN-L)
- 1368-423X
- OpenAlex
- W216325278
- Cited by
- 385
- References
- 93
- FWCI
- 36.03
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.