The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Settling the Polynomial Learnability of Mixtures of Gaussians
Abstract
Given data drawn from a mixture of multivariate Gaussians, a basic problem is to accurately estimate the mixture parameters. We give an algorithm for this problem that has running time and data requirements polynomial in the dimension and the inverse of the desired accuracy, with provably minimal assumptions on the Gaussians. As a simple consequence of our learning algorithm, we we give the first polynomial time algorithm for proper density estimation for mixtures of k Gaussians that needs no assumptions on the mixture. It was open whether proper density estimation was even statistically possible (with no assumptions) given only polynomially many samples, let alone whether it could be computationally efficient. The building blocks of our algorithm are based on the work (Kalai et al, STOC 2010) that gives an efficient algorithm for learning mixtures of two Gaussians by considering a series of projections down to one dimension, and applying the method of moments to each univariate projection. A major technical hurdle in the previous work is showing that one can efficiently learn univariate mixtures of two Gaussians. In contrast, because pathological scenarios can arise when considering projections of mixtures of more than two Gaussians, the bulk of the work in this paper concerns how to leverage a weaker algorithm for learning univariate mixtures (of many Gaussians) to learn in high dimensions. Our algorithm employs hierarchical clustering and rescaling, together with methods for backtracking and recovering from the failures that can occur in our univariate algorithm. Finally, while the running time and data requirements of our algorithm depend exponentially on the number of Gaussians in the mixture, we prove that such a dependence is necessary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
McLachlan, G., & Peel, D. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471721182 · Cited by 10,991
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
On the Convergence Properties of the EM Algorithm
Wu, C. F. J. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346060 · Cited by 3,269
Statistical Analysis of Finite Mixture Distributions.
Lindsay, B. G., Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1987) · Journal of the American Statistical Association · doi:10.2307/2289504 · Cited by 2,905
Mixture Densities, Maximum Likelihood and the EM Algorithm
Redner, R. A., & Walker, H. F. (1984) · SIAM Review · doi:10.1137/1026034 · Cited by 2,693
Statistical Analysis of Finite Mixture Distributions.
Newton, J., Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1986) · Biometrics · doi:10.2307/2531224 · Cited by 1,986
III. Contributions to the mathematical theory of evolution
Pearson, K. (1894) · Philosophical Transactions of the Royal Society of London. (A.) · doi:10.1098/rsta.1894.0003 · Cited by 1,550
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
Mixture Models: Theory, Geometry and Applications
Lindsay, B. G. (1995) · NSF-CBMS Regional Conference Series in Probability and Statistics · doi:10.1214/cbms/1462106013 · Cited by 1,084
Learning mixtures of Gaussians
Dasgupta, S. (2003) · 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039) · doi:10.1109/sffcs.1999.814639 · Cited by 588
Random Vectors in the Isotropic Position
Rudelson, M. (1999) · Journal of Functional Analysis · doi:10.1006/jfan.1998.3384 · Cited by 371
Teicher, H. (1961) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177705155 · Cited by 346
On the learnability of discrete distributions
Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R. E., & Sellie, L. (1994) · Proceedings of the twenty-sixth annual ACM symposium on Theory of computing - STOC '94 · doi:10.1145/195058.195155 · Cited by 273
On Spectral Learning of Mixtures of Distributions
Achlioptas, D., & McSherry, F. (2005) · Lecture notes in computer science · doi:10.1007/11503415_31 · Cited by 237
Learning mixtures of arbitrary gaussians
Sanjeev, A., & Kannan, R. (2001) · Proceedings of the thirty-third annual ACM symposium on Theory of computing · doi:10.1145/380752.380808 · Cited by 216
A spectral algorithm for learning mixture models
Vempala, S., & Wang, G. J. (2004) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(03)00180-6 · Cited by 216
Analysis of Perceptron-Based Active Learning
Dasgupta, S., Kalai, A. T., & Monteleoni, C. (2005) · Lecture Notes in Computer Science · doi:10.1007/11503415_17 · Cited by 214
Efficiently learning mixtures of two Gaussians
Kalai, A. T., Moitra, A., & Valiant, G. J. (2010) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/1806689.1806765 · Cited by 165
Self-citationA Two-round Variant of EM for Gaussian MixturesPreprint
Dasgupta, S., & Schulman, L. J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3850 · Cited by 141
Polynomial Learning of Distribution FamiliesPreprint
Belkin, M., & Sinha, K. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1004.4864 · Cited by 122
Learning Mixtures of Product Distributions using Correlations and Independence
Chaudhuri, K., & Rao, S. B. (n.d.) · Cited by 103
Isotropic PCA and Affine-Invariant Clustering
Brubaker, S. C., & Vempala, S. (2008) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2008.48 · Cited by 98
The Spectral Method for General Mixture Models
Kannan, R., Salmasian, H., & Vempala, S. (2008) · SIAM Journal on Computing · doi:10.1137/s0097539704445925 · Cited by 72
PAC Learning Axis-Aligned Mixtures of Gaussians with No Separation Assumption
Feldman, J., Servedio, R. A., & O’Donnell, R. (2006) · Lecture notes in computer science · doi:10.1007/11776420_5 · Cited by 71
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
Tensor decompositions for learning latent variable models
(2012) · arXiv (Cornell University) · doi:10.5555/2627435.2697055 · Cited by 779
“This was employed for parameter estimation of discrete Markov models [Cha96] using pair-wise and triple-wise probability tables, and later for other latent variable models such as hidden Markov models (HMMs), latent trees, Gaussian mixture models, and topic models such as latent Dirichlet allocation (LDA) [MR06, HKZ12…”
BackgroundInfluentialBlum, A., Hopcroft, J., & Kannan, R. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108755528 · Cited by 304
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231
Metadata
- Authors
- Moitra, A., & Valiant, G.
- Venue
- 2010 IEEE 51st Annual Symposium on Foundations of Computer Science
- Publisher
- IEEE
- Published
- 2010-10
- Pages
- 93-102
- ISSN
- 0272-5428 (ISSN-L)
- OpenAlex
- W2026302946
- Cited by
- 252
- References
- 26
- FWCI
- 9.10
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.