Cited by 430Machine Learning and AlgorithmsTopological and Geometric Data AnalysisGeometric Analysis and Curvature Flows

Testing the manifold hypothesis

Open access
Fefferman, Charles, Mitter, Sanjoy, Narayanan, Hariharan·2016·Journal of the American Mathematical Society

Abstract

The hypothesis that high dimensional data tend to lie in the vicinity of a low dimensional manifold is the basis of manifold learning. The goal of this paper is to develop an algorithm (with accompanying complexity guarantees) for testing the existence of a manifold that fits a probability distribution supported in a separable Hilbert space, only using i.i.d. samples from that distribution. More precisely, our setting is the following. Suppose that data are drawn independently at random from a probability distribution P supported on the unit ball of a separable Hilbert space H . Let G ( d , V , τ ) be the set of submanifolds of the unit ball of H whose volume is at most V and reach (which is the supremum of all r such that any point at a distance less than r has a unique nearest point on the manifold) is at least τ . Let L ( M , P ) denote the mean-squared distance of a random point from the probability distribution P to M . We obtain an algorithm that tests the manifold hypothesis in the following sense. The algorithm takes i.i.d. random samples from P as input and determines which of the following two is true (at least one must be): There exists M ∈ G ( d , C V , τ C ) such that L ( M , P ) ≤ C ϵ . There exists no M ∈ G ( d , V / C , C τ ) such that L ( M , P ) ≤ ϵ C . The answer is correct with probability at least 1 − δ .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1090/jams/852

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • According to the manifold hypothesis, most natural data lies on manifolds with significantly reduced intrinsic dimensionality.

    “According to the manifold hypothesis [66], most natural data lies on manifolds with significantly reduced intrinsic dimensionality.”
  • Data with manifold structures are ubiquitous in machine learning; the manifold hypothesis posits that natural data often reside on manifolds with lower intrinsic dimensionality.

    “Data with manifold structures are ubiquitous in machine learning.As the manifold hypothesis [66] posits, natural data often reside on manifolds with lower intrinsic dimensionality.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. Shun-ichi Amari and Hiroshi Nagaoka, Methods of information geometry, Translations of Mathematical Monographs, vol. 191, American Mathematical Society, Providence, RI; Oxford University Press, Oxford, 2000. Translated from the 1993 Japanese original by Daishi Harada. MR1800071 (2001j:62023)
  2. Richard G. Baraniuk and Michael B. Wakin, Random projections of smooth manifolds, Found. Comput. Math. 9 (2009), no. 1, 51-77, DOI 10.1007/s10208-007-9011-z. MR2472287 (2010g:94021) · DOI
  3. M. Brand, Charting a manifold, NIPS 15 (2002), 985-992.
  4. Guillermo D. Cañas, Tomaso Poggio, and Lorenzo Rosasco, Learning manifolds with k-means and k-flats, 26th Annual Conference on Neural Information Processing Systems, December 3-6, 2012, Lake Tahoe, Nevada, Proceedings of Advances in Neural Information Processing Systems 25, Curran Associates, Red Hook, NY, 2013, pp. 2474-2482.
  5. Siu-Wing Cheng, Tamal K. Dey, and Edgar A. Ramos, Manifold reconstruction from point samples, Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algo- rithms, ACM, New York, 2005, pp. 1018-1027. MR2298361
  6. Ronald R. Coifman, Stephane Lafon, Ann Lee, Mauro Maggioni, Boaz Nadler, Frederick Warner, and Steven Zucker, Geometric diffusions as a tool for harmonic analysis and struc- ture definition of data. Part II: Multiscale methods, Proc. Natl. Acad. Sci. USA 102 (2005), no. 21, 7432-7438.
  7. Trevor F. Cox and Michael A. A. Cox, Multidimensional scaling, Monographs on Statistics and Applied Probability, vol. 59, Chapman & Hall, London, 1994. With 1 IBM-PC floppy disk (3.5 inch, HD). MR1335449 (96g:62118)
  8. Guy David and Stephen Semmes, Analysis of and on uniformly rectifiable sets, Mathematical Surveys and Monographs, vol. 38, American Mathematical Society, Providence, RI, 1993. MR1251061 (94i:28003)
  9. Luc Devroye, Lászlo Györfi, and Gábor Lugosi, A probabilistic theory of pattern recognition, Springer, New York, 1996.
  10. Herbert Federer, Curvature measures, Trans. Amer. Math. Soc. 93 (1959), 418-491. MR0110078 (22 #961)
  11. H. Hotelling, Analysis of a complex of statistical variables into principal components, J. Educational Psychology 24 (1933), 417-441, 498-520.
  12. William B. Johnson and Joram Lindenstrauss, Extensions of Lipschitz mappings into a Hilbert space, Conference in modern analysis and probability (New Haven, Conn., 1982), Contemp. Math., vol. 26, Amer. Math. Soc., Providence, RI, 1984, pp. 189-206, DOI 10.1090/conm/026/737400. MR737400 (86a:46018) · DOI
  13. A. Kambhatla and Todd K. Leen, Fast non-linear dimension reduction, IEEE International Conference on Neural Networks, IEEE, New York, 1993, pp. 1213-1218.
  14. Michel Ledoux, The concentration of measure phenomenon, Mathematical Surveys and Monographs, vol. 89, American Mathematical Society, Providence, RI, 2001. MR1849347 (2003k:28019)
  15. Manifold learning theory and applications, CRC Press, Boca Raton, FL, 2012.
  16. Hariharan Narayanan and Sanjoy Mitter, On the sample complexity of testing the manifold hypothesis, NIPS (2010), 1786-1794.
  17. Hariharan Narayanan and Partha Niyogi, On the sample complexity of learning smooth cuts on a manifol, Proc. of the 22nd Annual Conference on Learning Theory (COLT), June 2009, pp. 197-204.
  18. Nigel J. Newton, Infinite-dimensional manifolds of finite-entropy probability measures, Geo- metric science of information, Lecture Notes in Comput. Sci., vol. 8085, Springer, Heidelberg, 2013, pp. 713-720, DOI 10.1007/978-3-642-40020-9 79. MR3126105 · DOI
  19. Umut Ozertem and Deniz Erdogmus, Locally defined principal curves and surfaces, J. Mach. Learn. Res. 12 (2011), 1249-1286. MR2804600 (2012e:62196)
  20. K. Pearson, On lines and planes of closest fit to systems of points in space, Philos. Mag. 2 (1901), 559-572.
  21. S. T. Roweis, L. Saul, and G. Hinton, Global coordination of local linear models, Adv. Neural Inf. Process. Syst. 14 (2002), 889-896.
  22. J. Shawe-Taylor and N. Christianini, Kernel methods for pattern analysis, Cambridge Uni- versity Press, Cambridge, UK, 2004.
  23. Alex J. Smola, Robert C. Williamson, Sebastian Mika, and Bernhard Schölkopf, Regularized principal manifolds, Computational learning theory (Nordkirchen, 1999), Lecture Notes in Comput. Sci., vol. 1572, Springer, Berlin, 1999, pp. 214-229, DOI 10.1007/3-540-49097-3 17. MR1724991 · DOI
  24. Pravin M. Vaidya, A new algorithm for minimizing convex functions over convex sets, Math. Programming 73 (1996), no. 3, Ser. A, 291-341, DOI 10.1016/0025-5610(92)00021- S. MR1393123 (97e:90060) · DOI

Cited by

Sorted by citations

Authors
Fefferman, C., Mitter, S., & Narayanan, H.
Affiliations
  • Princeton University (US)
  • Massachusetts Institute of Technology (US)
  • University of Washington (US)
Venue
Journal of the American Mathematical Society
Published
2016-02-09
Volume
29
Issue
4
Pages
983-1049
ISSN
  • 0894-0347 (ISSN-L)
  • 1088-6834
OpenAlex
W1593775104
Cited by
430
References
11
Citations by year
2026: 50 · 2025: 72 · 2024: 57 · 2023: 63 · 2022: 49 · 2021: 62 · 2020: 34 · 2019: 29 · 2018: 8 · 2017: 3 · 2015: 2 · 2014: 1
FWCI
4.20
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Mathematical Society (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (EECS -1135843 " Smart Power Systems of the Future : Foundations for Understanding Volatility and Improving Operational Reliabili)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Topological and Geometric Data Analysis
  • Physical Sciences › Mathematics › Applied Mathematics › Geometric Analysis and Curvature Flows
Keywords
manifold hypothesis, manifold learning, separable Hilbert space, i.i.d. samples, high-dimensional data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review