The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Heteroskedastic PCA: Algorithm, optimality, and applications
Abstract
A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is computationally efficient and provably optimal under the generalized spiked covariance model. A key technical step is a deterministic robust perturbation analysis on singular subspaces, which can be of independent interest. The effectiveness of the proposed algorithm is demonstrated in a suite of problems in high-dimensional statistics, including singular value decomposition (SVD) under heteroskedastic noise, Poisson PCA, and SVD for heteroskedastic and incomplete data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deleting the diagonal of a Gram matrix is an established device for heteroskedastic data (Cai et al., 2021; Zhang et al., 2022).”
From The Limits of Query Monitoring against Distributed Model Extraction (2026) · cites this paper as Zhang et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fortunato, S. (2010) · Physics Reports · doi:10.1016/j.physrep.2009.11.002 · Cited by 11,293 · cited 2×
“Fortunato, S. (2010). Community detection in graphs.”
BackgroundProbabilistic Principal Component Analysis
Tipping, M. E., & Bishop, C. M. (1999) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00196 · Cited by 3,679
Concentration Inequalities: A Nonasymptotic Theory of Independence
Boucheron, S., Lugosi, G., & Massart, P. (2013) · doi:10.1093/acprof:oso/9780199535255.001.0001 · Cited by 1,768
Concentration Inequalities and Model Selection
(2007) · Lecture Notes in Mathematics · doi:10.1007/978-3-540-48503-2 · Cited by 744
Research works printed but not linked: 55
- Hotelling, H. (1936). Relations between two sets of variates. Biometrika 28 321–377. · DOI · Paper page
- Candès, E. J., Sing-Long, C. A. and Trzasko, J. D. (2013). Unbiased risk estimates for singular value thresholding and spectral estimators. IEEE Trans. Signal Process. 61 4643–4657. · DOI · Paper page
- Bubeck, S. (2015). Convex optimization: Algorithms and complexity. Found. Trends Mach. Learn. 8 231–357. · DOI · Paper page
- Jain, P., Netrapalli, P. and Sanghavi, S. (2013). Low-rank matrix completion using alternating minimization (extended abstract). In STOC’13—Proceedings of the 2013 ACM Symposium on Theory of Computing 665–674. ACM, New York. · DOI · Paper page
- Donoho, D. and Gavish, M. (2014). Minimax risk of matrix denoising by singular value thresholding. Ann. Statist. 42 2413–2440. · DOI
- CAO, Y., ZHANG, A. and LI, H. (2020). Multisample estimation of bacterial composition matrices in metagenomics data. Biometrika 107 75–92. · DOI
- SALMON, J., HARMANY, Z., DELEDALLE, C.-A. and WILLETT, R. (2014). Poisson noise reduction with non-local PCA. J. Math. Imaging Vision 48 279–294. · DOI · Paper page
- BARTLETT, M. S. (1937). The statistical conception of mental factors. Br. J. Health Psychol. 28 97. · DOI · Paper page
- CAI, J.-F. and WEI, K. (2018). Exploiting the structure effectively and efficiently in low-rank matrix recovery. In Processing, Analyzing and Learning of Images, Shapes, and Forms. Part 1. Handb. Numer. Anal. 19 21–51. North-Holland, Amsterdam. · DOI
- CAI, T. T., HAN, R. and ZHANG, A. R. (2020). On the non-asymptotic concentration of heteroskedastic Wishart-type matrix. ArXiv preprint. Available at arXiv:2008.12434. · arXiv
- CAI, T. T. and ZHANG, A. (2018). Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics. Ann. Statist. 46 60–89. · DOI
- COLLINS, M., DASGUPTA, S. and SCHAPIRE, R. E. (2002). A generalization of principal components analysis to the exponential family. In Advances in Neural Information Processing Systems 617–624. · DOI
- DE LATHAUWER, L., DE MOOR, B. and VANDEWALLE, J. (2000). On the best rank-1 and rank-(R1,R2,…,RN) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21 1324–1342. · DOI · Paper page
- FLORESCU, L. and PERKINS, W. (2016). Spectral thresholds in the bipartite stochastic block model. In Conference on Learning Theory 943–959.
- GAVISH, M. and DONOHO, D. L. (2017). Optimal shrinkage of singular values. IEEE Trans. Inf. Theory 63 2137–2152. · DOI · Paper page
- GHOSH, J. and DUNSON, D. B. (2009). Default prior distributions and efficient posterior computation in Bayesian factor analysis. J. Comput. Graph. Statist. 18 306–320. · DOI
- HONG, D., BALZANO, L. and FESSLER, J. A. (2016). Towards a theoretical analysis of PCA for heteroscedastic data. In Communication, Control, and Computing (Allerton), 2016 54 th Annual Allerton Conference on 496–503. IEEE, Los Alamitos. · doi:10.1109/allerton.2016.7852272
- HONG, D., BALZANO, L. and FESSLER, J. A. (2018). Asymptotic performance of PCA for high-dimensional heteroscedastic data. J. Multivariate Anal. 167 435–452. · DOI
- HONG, D., BALZANO, L. and FESSLER, J. A. (2018). Asymptotic performance of PCA for high-dimensional heteroscedastic data. J. Multivariate Anal. 167 435–452. · DOI
- KESHAVAN, R. H. (2012). Efficient algorithms for collaborative filtering Ph.D. thesis Stanford Univ.
- LAWLEY, D. N. and MAXWELL, A. E. (1962). Factor analysis as a statistical method. J. R. Stat. Soc., Ser. D, Stat. 12 209–229. · DOI · Paper page
- LIU, L. T., DOBRIBAN, E. and SINGER, A. (2018). e PCA: High dimensional exponential family PCA. Ann. Appl. Stat. 12 2121–2150. · DOI
- LUO, Y., HAN, R. and ZHANG, A. R. (2020). A Schatten-q matrix perturbation theory via perturbation projection error bound. ArXiv preprint. Available at arXiv:2008.01312. · arXiv
- MARTIN, A. D., QUINN, K. M. and PARK, J. H. (2011). MCMCpack: Markov chain Monte Carlo in R. J. Stat. Softw. 42. · DOI · Paper page
- MELAMED, D. (2014). Community structures in bipartite networks: A dual-projection approach. PLoS ONE 9 e97823. · DOI
- MOHAMED, S., GHAHRAMANI, Z. and HELLER, K. A. (2009). Bayesian exponential family PCA. In Advances in Neural Information Processing Systems 1089–1096.
- OWEN, A. B. and WANG, J. (2016). Bi-cross-validation for factor analysis. Statist. Sci. 31 119–139. · DOI
- ROBIN, G., JOSSE, J., MOULINES, É. and SARDY, S. (2019). Low-rank model with covariates for count data with missing values. J. Multivariate Anal. 173 416–434. · DOI
- THOMSON, G. (1939). The factorial analysis of human ability. Br. J. Educ. Psychol. 9 188–195. · DOI · Paper page
- VASWANI, N. and GUO, H. (2016). Correlated-PCA: Principal components’ analysis when data and noise are correlated. In Advances in Neural Information Processing Systems 1768–1776.
- VASWANI, N. and NARAYANAMURTHY, P. (2017). Finite sample guarantees for PCA in non-isotropic and data-dependent noise. In 2017 55 th Annual Allerton Conference on Communication, Control, and Computing (Allerton) 783–789. IEEE, Los Alamitos. · doi:10.1109/allerton.2017.8262819
- VASWANI, N. and NARAYANAMURTHY, P. (2018). PCA in sparse data-dependent noise. In 2018 IEEE International Symposium on Information Theory (ISIT) 641–645. IEEE, Los Alamitos. · doi:10.1109/isit.2018.8437331
- VASWANI, N. and NARAYANAMURTHY, P. (2020). Fast robust subspace tracking via PCA in sparse data-dependent noise. IEEE J. Sel. Areas Inf. Theory 1 723–744. · doi:10.1109/jsait.2020.3040365
- ZHANG, A., CAI, T. T. and WU, Y. (2022). Supplement to “Heteroskedastic PCA: Algorithm, optimality, and applications.” https://doi.org/10.1214/21-AOS2074SUPP · doi:10.1214/21-aos2074supp
- Candès, E. J. and Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56 2053–2080. · DOI · Paper page
- Bai, J. and Li, K. (2012). Statistical analysis of factor models of high dimension. Ann. Statist. 40 436–465. · DOI · Paper page
- Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643–1697. · DOI · Paper page
- Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Found. Comput. Math. 9 717–772. · DOI · Paper page
- Davis, C. and Kahan, W. M. (1970). The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal. 7 1–46. · DOI · Paper page
- Donoho, D., Gavish, M. and Johnstone, I. (2018). Optimal shrinkage of eigenvalues in the spiked covariance model. Ann. Statist. 46 1742–1778. · DOI · Paper page
- Wang, W. and Fan, J. (2017). Asymptotics of empirical eigenstructure for high dimensional spiked covariance. Ann. Statist. 45 1342–1374. · DOI · Paper page
- Wedin, P. (1972). Perturbation bounds in connection with singular value decomposition. BIT 12 99–111. · DOI · Paper page
- Yu, Y., Wang, T. and Samworth, R. J. (2015). A useful variant of the Davis–Kahan theorem for statisticians. Biometrika 102 315–323. · DOI · Paper page
- Chatterjee, S. (2015). Matrix estimation by universal singular value thresholding. Ann. Statist. 43 177–214. · DOI · Paper page
- Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167–177. · DOI
- Cai, T. T. and Zhang, A. (2016). Minimax rate-optimal estimation of high-dimensional covariance matrices with incomplete data. J. Multivariate Anal. 150 55–74. · DOI
- Keshavan, R. H., Montanari, A. and Oh, S. (2010). Matrix completion from a few entries. IEEE Trans. Inf. Theory 56 2980–2998. · DOI · Paper page
- Lounici, K. (2014). High-dimensional covariance matrix estimation with missing observations. Bernoulli 20 1029–1058. · DOI · Paper page
- Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791–2817. · DOI · Paper page
- Zhang, A. and Xia, D. (2018). Tensor SVD: Statistical and computational limits. IEEE Trans. Inf. Theory 64 7311–7338. · DOI · Paper page
- Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge Univ. Press, Cambridge. · DOI · Paper page
- Nadakuditi, R. R. (2014). OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage. IEEE Trans. Inf. Theory 60 3002–3018. · DOI · Paper page
- Baik, J. and Silverstein, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382–1408. · DOI · Paper page
- Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327. · DOI · Paper page
- Yao, J., Zheng, S. and Bai, Z. (2015). Large Sample Covariance Matrices and High-Dimensional Data Analysis. Cambridge Series in Statistical and Probabilistic Mathematics 39. Cambridge Univ. Press, New York. · DOI · Paper page
Cited by
Subspace Estimation from Unbalanced and Incomplete Data Matrices: 𝓁2, ∞ Statistical Guarantees
Cai, C., Li, G., Chi, Y., Poor, H. V., & Chen, Y. (2021) · The Annals of Statistics · doi:10.1214/20-aos1986 · Cited by 37
Biwhitening Reveals the Rank of a Count Matrix
Landa, B., Zhang, T. T. C. K., & Kluger, Y. (2022) · SIAM Journal on Mathematics of Data Science · doi:10.1137/21m1456807 · Cited by 20
“A large portion of existing literature on PCA in heteroskedastic noise is dedicated to the task of estimating X or the subspaces spanning its rows and columns [35, 74, 34, 53, 51, 54, 12, 22].”
BackgroundMethodMelsen, K. A. C., Kunst, J. F., Crossa, J., Krause, M. R., van Eeuwijk, F. A., Kruijer, W., & Peeters, C. F. W. (2025) · Biometrical Journal · doi:10.1002/bimj.70081 · Cited by 2
The Limits of Query Monitoring against Distributed Model Extraction
Khilar, S. C. (2026) · arXiv · Cited by 1
Metadata
- Authors
- Zhang, A. R., Cai, T. T., & Wu, Y.
- Affiliations
- Department of Statistics, University of Wisconsin-Madison
- Department of Statistics, The Wharton School, University of Pennsylvania
- Department of Statistics and Data Science, Yale University
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2022-02-01
- Volume
- 50
- Issue
- 1
- Pages
- 53-80
- ISSN
- 0090-5364
- 2168-8966
- Cited by
- 52
- References
- 4
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Heteroskedastic PCA: Algorithm, optimality, and applications. https://app.cochat.ai/papers/10.1214/21-aos2074
- The paper
- Zhang, A. R., Cai, T. T., & Wu, Y. (2022). Heteroskedastic PCA: Algorithm, optimality, and applications. The Annals of Statistics. https://doi.org/10.1214/21-aos2074
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.