CoChat
Cited by 52

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Heteroskedastic PCA: Algorithm, optimality, and applications

Zhang, Anru R., Cai, T. Tony, Wu, Yihong · 2022 · The Annals of Statistics

Abstract

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is computationally efficient and provably optimal under the generalized spiked covariance model. A key technical step is a deterministic robust perturbation analysis on singular subspaces, which can be of independent interest. The effectiveness of the proposed algorithm is demonstrated in a suite of problems in high-dimensional statistics, including singular value decomposition (SVD) under heteroskedastic noise, Poisson PCA, and SVD for heteroskedastic and incomplete data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1214/21-aos2074

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 4 references

Research works printed but not linked: 55

  1. Hotelling, H. (1936). Relations between two sets of variates. Biometrika 28 321–377. · DOI · Paper page
  2. Candès, E. J., Sing-Long, C. A. and Trzasko, J. D. (2013). Unbiased risk estimates for singular value thresholding and spectral estimators. IEEE Trans. Signal Process. 61 4643–4657. · DOI · Paper page
  3. Bubeck, S. (2015). Convex optimization: Algorithms and complexity. Found. Trends Mach. Learn. 8 231–357. · DOI · Paper page
  4. Jain, P., Netrapalli, P. and Sanghavi, S. (2013). Low-rank matrix completion using alternating minimization (extended abstract). In STOC’13—Proceedings of the 2013 ACM Symposium on Theory of Computing 665–674. ACM, New York. · DOI · Paper page
  5. Donoho, D. and Gavish, M. (2014). Minimax risk of matrix denoising by singular value thresholding. Ann. Statist. 42 2413–2440. · DOI
  6. CAO, Y., ZHANG, A. and LI, H. (2020). Multisample estimation of bacterial composition matrices in metagenomics data. Biometrika 107 75–92. · DOI
  7. SALMON, J., HARMANY, Z., DELEDALLE, C.-A. and WILLETT, R. (2014). Poisson noise reduction with non-local PCA. J. Math. Imaging Vision 48 279–294. · DOI · Paper page
  8. BARTLETT, M. S. (1937). The statistical conception of mental factors. Br. J. Health Psychol. 28 97. · DOI · Paper page
  9. CAI, J.-F. and WEI, K. (2018). Exploiting the structure effectively and efficiently in low-rank matrix recovery. In Processing, Analyzing and Learning of Images, Shapes, and Forms. Part 1. Handb. Numer. Anal. 19 21–51. North-Holland, Amsterdam. · DOI
  10. CAI, T. T., HAN, R. and ZHANG, A. R. (2020). On the non-asymptotic concentration of heteroskedastic Wishart-type matrix. ArXiv preprint. Available at arXiv:2008.12434. · arXiv
  11. CAI, T. T. and ZHANG, A. (2018). Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics. Ann. Statist. 46 60–89. · DOI
  12. COLLINS, M., DASGUPTA, S. and SCHAPIRE, R. E. (2002). A generalization of principal components analysis to the exponential family. In Advances in Neural Information Processing Systems 617–624. · DOI
  13. DE LATHAUWER, L., DE MOOR, B. and VANDEWALLE, J. (2000). On the best rank-1 and rank-(R1,R2,…,RN) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21 1324–1342. · DOI · Paper page
  14. FLORESCU, L. and PERKINS, W. (2016). Spectral thresholds in the bipartite stochastic block model. In Conference on Learning Theory 943–959.
  15. GAVISH, M. and DONOHO, D. L. (2017). Optimal shrinkage of singular values. IEEE Trans. Inf. Theory 63 2137–2152. · DOI · Paper page
  16. GHOSH, J. and DUNSON, D. B. (2009). Default prior distributions and efficient posterior computation in Bayesian factor analysis. J. Comput. Graph. Statist. 18 306–320. · DOI
  17. HONG, D., BALZANO, L. and FESSLER, J. A. (2016). Towards a theoretical analysis of PCA for heteroscedastic data. In Communication, Control, and Computing (Allerton), 2016 54 th Annual Allerton Conference on 496–503. IEEE, Los Alamitos. · doi:10.1109/allerton.2016.7852272
  18. HONG, D., BALZANO, L. and FESSLER, J. A. (2018). Asymptotic performance of PCA for high-dimensional heteroscedastic data. J. Multivariate Anal. 167 435–452. · DOI
  19. HONG, D., BALZANO, L. and FESSLER, J. A. (2018). Asymptotic performance of PCA for high-dimensional heteroscedastic data. J. Multivariate Anal. 167 435–452. · DOI
  20. KESHAVAN, R. H. (2012). Efficient algorithms for collaborative filtering Ph.D. thesis Stanford Univ.
  21. LAWLEY, D. N. and MAXWELL, A. E. (1962). Factor analysis as a statistical method. J. R. Stat. Soc., Ser. D, Stat. 12 209–229. · DOI · Paper page
  22. LIU, L. T., DOBRIBAN, E. and SINGER, A. (2018). e PCA: High dimensional exponential family PCA. Ann. Appl. Stat. 12 2121–2150. · DOI
  23. LUO, Y., HAN, R. and ZHANG, A. R. (2020). A Schatten-q matrix perturbation theory via perturbation projection error bound. ArXiv preprint. Available at arXiv:2008.01312. · arXiv
  24. MARTIN, A. D., QUINN, K. M. and PARK, J. H. (2011). MCMCpack: Markov chain Monte Carlo in R. J. Stat. Softw. 42. · DOI · Paper page
  25. MELAMED, D. (2014). Community structures in bipartite networks: A dual-projection approach. PLoS ONE 9 e97823. · DOI
  26. MOHAMED, S., GHAHRAMANI, Z. and HELLER, K. A. (2009). Bayesian exponential family PCA. In Advances in Neural Information Processing Systems 1089–1096.
  27. OWEN, A. B. and WANG, J. (2016). Bi-cross-validation for factor analysis. Statist. Sci. 31 119–139. · DOI
  28. ROBIN, G., JOSSE, J., MOULINES, É. and SARDY, S. (2019). Low-rank model with covariates for count data with missing values. J. Multivariate Anal. 173 416–434. · DOI
  29. THOMSON, G. (1939). The factorial analysis of human ability. Br. J. Educ. Psychol. 9 188–195. · DOI · Paper page
  30. VASWANI, N. and GUO, H. (2016). Correlated-PCA: Principal components’ analysis when data and noise are correlated. In Advances in Neural Information Processing Systems 1768–1776.
  31. VASWANI, N. and NARAYANAMURTHY, P. (2017). Finite sample guarantees for PCA in non-isotropic and data-dependent noise. In 2017 55 th Annual Allerton Conference on Communication, Control, and Computing (Allerton) 783–789. IEEE, Los Alamitos. · doi:10.1109/allerton.2017.8262819
  32. VASWANI, N. and NARAYANAMURTHY, P. (2018). PCA in sparse data-dependent noise. In 2018 IEEE International Symposium on Information Theory (ISIT) 641–645. IEEE, Los Alamitos. · doi:10.1109/isit.2018.8437331
  33. VASWANI, N. and NARAYANAMURTHY, P. (2020). Fast robust subspace tracking via PCA in sparse data-dependent noise. IEEE J. Sel. Areas Inf. Theory 1 723–744. · doi:10.1109/jsait.2020.3040365
  34. ZHANG, A., CAI, T. T. and WU, Y. (2022). Supplement to “Heteroskedastic PCA: Algorithm, optimality, and applications.” https://doi.org/10.1214/21-AOS2074SUPP · doi:10.1214/21-aos2074supp
  35. Candès, E. J. and Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56 2053–2080. · DOI · Paper page
  36. Bai, J. and Li, K. (2012). Statistical analysis of factor models of high dimension. Ann. Statist. 40 436–465. · DOI · Paper page
  37. Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643–1697. · DOI · Paper page
  38. Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Found. Comput. Math. 9 717–772. · DOI · Paper page
  39. Davis, C. and Kahan, W. M. (1970). The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal. 7 1–46. · DOI · Paper page
  40. Donoho, D., Gavish, M. and Johnstone, I. (2018). Optimal shrinkage of eigenvalues in the spiked covariance model. Ann. Statist. 46 1742–1778. · DOI · Paper page
  41. Wang, W. and Fan, J. (2017). Asymptotics of empirical eigenstructure for high dimensional spiked covariance. Ann. Statist. 45 1342–1374. · DOI · Paper page
  42. Wedin, P. (1972). Perturbation bounds in connection with singular value decomposition. BIT 12 99–111. · DOI · Paper page
  43. Yu, Y., Wang, T. and Samworth, R. J. (2015). A useful variant of the Davis–Kahan theorem for statisticians. Biometrika 102 315–323. · DOI · Paper page
  44. Chatterjee, S. (2015). Matrix estimation by universal singular value thresholding. Ann. Statist. 43 177–214. · DOI · Paper page
  45. Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167–177. · DOI
  46. Cai, T. T. and Zhang, A. (2016). Minimax rate-optimal estimation of high-dimensional covariance matrices with incomplete data. J. Multivariate Anal. 150 55–74. · DOI
  47. Keshavan, R. H., Montanari, A. and Oh, S. (2010). Matrix completion from a few entries. IEEE Trans. Inf. Theory 56 2980–2998. · DOI · Paper page
  48. Lounici, K. (2014). High-dimensional covariance matrix estimation with missing observations. Bernoulli 20 1029–1058. · DOI · Paper page
  49. Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791–2817. · DOI · Paper page
  50. Zhang, A. and Xia, D. (2018). Tensor SVD: Statistical and computational limits. IEEE Trans. Inf. Theory 64 7311–7338. · DOI · Paper page
  51. Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge Univ. Press, Cambridge. · DOI · Paper page
  52. Nadakuditi, R. R. (2014). OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage. IEEE Trans. Inf. Theory 60 3002–3018. · DOI · Paper page
  53. Baik, J. and Silverstein, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382–1408. · DOI · Paper page
  54. Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327. · DOI · Paper page
  55. Yao, J., Zheng, S. and Bai, Z. (2015). Large Sample Covariance Matrices and High-Dimensional Data Analysis. Cambridge Series in Statistical and Probabilistic Mathematics 39. Cambridge Univ. Press, New York. · DOI · Paper page

Cited by

See all 52

Metadata

Authors
Zhang, A. R., Cai, T. T., & Wu, Y.
Affiliations
  • Department of Statistics, University of Wisconsin-Madison
  • Department of Statistics, The Wharton School, University of Pennsylvania
  • Department of Statistics and Data Science, Yale University
Venue
The Annals of Statistics
Published
2022-02-01
Volume
50
Issue
1
Pages
53-80
ISSN
  • 0090-5364
  • 2168-8966
Cited by
52
References
4

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Heteroskedastic PCA: Algorithm, optimality, and applications. https://app.cochat.ai/papers/10.1214/21-aos2074
The paper
Zhang, A. R., Cai, T. T., & Wu, Y. (2022). Heteroskedastic PCA: Algorithm, optimality, and applications. The Annals of Statistics. https://doi.org/10.1214/21-aos2074

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.