Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
Open accessAbstract
We provide a framework for structural multiscale geometric organization of graphs and subsets of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \begin{equation*}{\mathbb{R}}^{n}\end{equation*}\end{document} . We use diffusion semigroups to generate multiscale geometries in order to organize and represent complex structures. We show that appropriately selected eigenfunctions or scaling functions of Markov matrices, which describe local transitions, lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. We provide a unified view of ideas from data analysis, machine learning, and numerical analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Diffusion maps try to link geometric data to the characteristic time scale of the system's motion.
“methods such as diffusion maps 9,77-79 try to link geometric data to the characteristic time scale of the system's motion.”
From Unsupervised machine learning in atomistic simulations, between predictions and understanding (2019) · cites this paper as [77]
PCA and diffusion maps are popular dimensionality reduction techniques that are principally summarization methods.
“Two popular dimensionality reduction techniques that are principally summarization methods are principal component analysis (PCA; Pearson, 1901) and diffusion maps (Coifman et al, 2005) which were popularized for single‐cell analysis by Haghverdi et al (2015).”
From Current best practices in single-cell RNA-seq analysis: a tutorial (2019) · cites this paper as Coifman et al, 2005
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,842 · see citing works
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Belkin, M., & Niyogi, P. (2003) · Neural Computation · doi:10.1162/089976603321780317 · Cited by 7,562 · see citing works
Chung, F. (1996) · Regional conference series in mathematics · doi:10.1090/cbms/092 · Cited by 5,918 · see citing works
Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data
Donoho, D. L., & Grimes, C. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1031596100 · Cited by 1,704 · see citing works
Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
Nadler, B., Lafon, S., Coifman, R. R., & Kevrekidis, I. G. (2006) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2005.07.004 · Cited by 989 · see citing works
Self-citationGear, C. W., Hyman, J. M., Kevrekidid, P. G., Kevrekidis, I. G., Runborg, O., & Theodoropoulos, C. (2003) · Communications in Mathematical Sciences · doi:10.4310/cms.2003.v1.n4.a5 · Cited by 778 · see citing works
Segmentation using eigenvectors: a unifying view
Weiss, Y. (1999) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1999.790354 · Cited by 729 · see citing works
A kernel view of the dimensionality reduction of manifolds
Ham, J., Lee, D. D., Mika, S., & Schölkopf, B. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015417 · Cited by 539 · see citing works
Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space AlignmentPreprint
Zhang, Z., & Zha, H. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212008 · Cited by 536 · see citing works
Coifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F. J., & Zucker, S. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500896102 · Cited by 217 · see citing works
Self-citation‘Coarse’ integration/bifurcation analysis via microscopic simulators: micro-Galerkin methods
Gear, C. W., Kevrekidis, I. G., & Theodoropoulos, C. (2002) · Computers & Chemical Engineering · doi:10.1016/s0098-1354(02)00020-0 · Cited by 199 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [11]Pedersen, K. S. & Lee, A. B. (2002) Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark , eds. Nielsen, M., Heyden, A., Sparr, G. & Johansen, P. (Springer, Berlin), pp. 328-342.
- [12]Huggins, P. S. & Zucker, S. W. (2002) Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark , eds. Nielsen, M., Heyden, A., Sparr, G. & Johansen, P. (Springer, Berlin), pp. 384-398.
- [13]Shi, J. & Malik, J. (2000) IEEE Trans. Pattern Anal. Machine Intell. 22, 888-905.
- [17]Szummer, M. & Jaakkola, T. (2001) Adv. Neural Inf. Process. Syst. 14, 945-952.
- [18]Coifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F. & Zucker, S. W. (2005) Proc. Natl. Acad. Sci. USA 102, 7432-7437.
Cited by
Sorted by citations
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881 · see citing works
Dimensionality reduction for visualizing single-cell data using UMAP
Becht, E., McInnes, L., Healy, J., Dutertre, C.-A., Kwok, I. W. H., Ng, L. G., Ginhoux, F., & Newell, E. W. (2019) · Nature Biotechnology · doi:10.1038/nbt.4314 · Cited by 5,358 · see citing works
Situating the default-mode network along a principal gradient of macroscale cortical organization
Margulies, D. S., Ghosh, S. S., Goulas, A., Falkiewicz, M., Huntenburg, J. M., Langs, G., Bezgin, G., Eickhoff, S. B., Castellanos, F. X., Petrides, M., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1608282113 · Cited by 2,321 · see citing works
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
Histopathological Image Analysis: A Review
Gurcan, M. N., Boucheron, L. E., Can, A., Madabhushi, A., Rajpoot, N. M., & Yener, B. (2009) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2009.2034865 · Cited by 2,052 · see citing works
A single-cell survey of the small intestinal epithelium
Haber, A. L., Biton, M., Rogel, N., Herbst, R. H., Shekhar, K., Smillie, C., Burgin, G., Delorey, T. M., Howitt, M. R., Katz, Y., et al. (2017) · Nature · doi:10.1038/nature24489 · Cited by 1,750 · see citing works
Wolf, F. A., Hamey, F. K., Plass, M., Solana, J., Dahlin, J. S., Göttgens, B., Rajewsky, N., Simon, L., & Theis, F. J. (2019) · Genome Biology · doi:10.1186/s13059-019-1663-x · Cited by 1,698 · see citing works
Diffusion pseudotime robustly reconstructs lineage branching
Haghverdi, L., Büttner, M., Wolf, F. A., Buettner, F., & Theis, F. J. (2016) · Nature Methods · doi:10.1038/nmeth.3971 · Cited by 1,385 · see citing works
ArchR is a scalable software package for integrative single-cell chromatin accessibility analysis
Granja, J. M., Corces, M. R., Pierce, S. E., Bagdatli, S. T., Choudhry, H., Chang, H. Y., & Greenleaf, W. J. (2021) · Nature Genetics · doi:10.1038/s41588-021-00790-6 · Cited by 1,392 · see citing works
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438 · see citing works
Identification of slow molecular order parameters for Markov model construction
Pérez-Hernández, G., Paul, F., Giorgino, T., De Fabritiis, G., & Noé, F. (2013) · The Journal of Chemical Physics · doi:10.1063/1.4811489 · Cited by 990 · see citing works
“The resulting models are often called transition networks [74, 62, 32], Diffusion maps [18, 76], Master equation models [87, 14], Markov models [64] or Markov state models [86, 16] (MSM), where “Markovianity” means that the kinetics are modeled by a memoryless jump process between states.”
BackgroundCharacterization of cell fate probabilities in single-cell data with Palantir
Setty, M., Kiseliovas, V., Levine, J., Gayoso, A., Mazutis, L., & Pe’er, D. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0068-4 · Cited by 833 · see citing works
Comparing molecules and solids across structural and alchemical space
De, S., Bartók, A. P., Csányi, G., & Ceriotti, M. (2016) · Physical Chemistry Chemical Physics · doi:10.1039/c6cp00415f · Cited by 778 · see citing works
A single-cell resolution map of mouse hematopoietic stem and progenitor cell differentiation
Nestorowa, S., Hamey, F. K., Pijuan Sala, B., Diamanti, E., Shepherd, M., Laurenti, E., Wilson, N. K., Kent, D. G., & Göttgens, B. (2016) · Blood · doi:10.1182/blood-2016-05-716480 · Cited by 768 · see citing works
Diffusion maps for high-dimensional single-cell analysis of differentiation data
Haghverdi, L., Buettner, F., & Theis, F. J. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv325 · Cited by 649 · see citing works
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632 · see citing works
Wishbone identifies bifurcating developmental trajectories from single-cell data
Setty, M., Tadmor, M. D., Reich-Zeliger, S., Angel, O., Salame, T. M., Kathail, P., Choi, K., Bendall, S., Friedman, N., & Pe'er, D. (2016) · Nature Biotechnology · doi:10.1038/nbt.3569 · Cited by 558 · see citing works
Global dynamics of proteins: bridging between structure and function
Bahar, I., Lezon, T. R., Yang, L.-W., & Eyal, E. (2010) · Annual Review of Biophysics · doi:10.1146/annurev.biophys.093008.131258 · Cited by 610 · see citing works
Moignard, V., Woodhouse, S., Haghverdi, L., Lilly, A. J., Tanaka, Y., Wilkinson, A. C., Buettner, F., Macaulay, I. C., Jawaid, W., Diamanti, E., et al. (2015) · Nature Biotechnology · doi:10.1038/nbt.3154 · Cited by 380 · see citing works
Fundamental limits on dynamic inference from single-cell snapshots
Weinreb, C., Wolock, S., Tusi, B. K., Socolovsky, M., & Klein, A. M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1714723115 · Cited by 341 · see citing works
Data Fusion and Multicue Data Matching by Diffusion Maps
Lafon, S., Keller, Y., & Coifman, R. R. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.223 · Cited by 312 · see citing works
Self-citationMachine learning approaches for analyzing and enhancing molecular dynamics simulations
Wang, Y., Lamim Ribeiro, J. M., & Tiwary, P. (2020) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2019.12.016 · Cited by 303 · see citing works
Determination of reaction coordinates via locally scaled diffusion map
Rohrdanz, M. A., Zheng, W., Maggioni, M., & Clementi, C. (2011) · The Journal of Chemical Physics · doi:10.1063/1.3569857 · Cited by 292 · see citing works
Self-citationBronstein, A. M., Bronstein, M. M., Kimmel, R., Mahmoudi, M., & Sapiro, G. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0301-6 · Cited by 267 · see citing works
A single-cell hematopoietic landscape resolves 8 lineage trajectories and defects in Kit mutant mice
Dahlin, J. S., Hamey, F. K., Pijuan-Sala, B., Shepherd, M., Lau, W. W. Y., Nestorowa, S., Weinreb, C., Wolock, S., Hannah, R., Diamanti, E., et al. (2018) · Blood · doi:10.1182/blood-2017-12-821413 · Cited by 233 · see citing works
Spectral gap optimization of order parameters for sampling complex molecular systems
Tiwary, P., & Berne, B. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1600917113 · Cited by 259 · see citing works
Rohrdanz, M. A., Zheng, W., & Clementi, C. (2013) · Annual Review of Physical Chemistry · doi:10.1146/annurev-physchem-040412-110006 · Cited by 242 · see citing works
Sun, S., Zhu, J., Ma, Y., & Zhou, X. (2019) · Genome biology · doi:10.1186/s13059-019-1898-6 · Cited by 225 · see citing works
Unsupervised machine learning in atomistic simulations, between predictions and understanding
Ceriotti, M. (2019) · The Journal of Chemical Physics · doi:10.1063/1.5091842 · Cited by 173 · see citing works
Reconstructing gene regulatory dynamics from high-dimensional single-cell snapshot data
Ocone, A., Haghverdi, L., Mueller, N. S., & Theis, F. J. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv257 · Cited by 155 · see citing works
Systematic determination of order parameters for chain dynamics using diffusion maps
Ferguson, A. L., Panagiotopoulos, A. Z., Debenedetti, P. G., & Kevrekidis, I. G. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1003293107 · Cited by 165 · see citing works
Fast and precise single-cell data analysis using a hierarchical autoencoder
Tran, D., Nguyen, H., Tran, B., La Vecchia, C., Luu, H. N., & Nguyen, T. (2021) · Nature Communications · doi:10.1038/s41467-021-21312-2 · Cited by 145 · see citing works
Topological data analysis of contagion maps for examining spreading processes on networks
Taylor, D., Klimm, F., Harrington, H. A., Kramár, M., Mischaikow, K., Porter, M. A., & Mucha, P. J. (2015) · Nature Communications · doi:10.1038/ncomms8723 · Cited by 148 · see citing works
Li, C., Virgilio, M. C., Collins, K. L., & Welch, J. D. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01476-y · Cited by 130 · see citing works
Intrinsic map dynamics exploration for uncharted effective free-energy landscapes
Chiavazzo, E., Covino, R., Coifman, R. R., Gear, C. W., Georgiou, A. S., Hummer, G., & Kevrekidis, I. G. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1621481114 · Cited by 125 · see citing works
Self-citationDynamic quantum clustering: A method for visual exploration of structures in data
Weinstein, M., & Horn, D. (2009) · Physical Review E · doi:10.1103/physreve.80.066117 · Cited by 51 · see citing works
Single cell-derived spheroids capture the self-renewing subpopulations of metastatic ovarian cancer
Velletri, T., Villa, C. E., Cilli, D., Barzaghi, B., Lo Riso, P., Lupia, M., Luongo, R., López-Tobón, A., De Simone, M., Bonnal, R. J. P., et al. (2022) · Cell Death & Differentiation · doi:10.1038/s41418-021-00878-w · Cited by 48 · see citing works
Mapping and classifying molecules from a high-throughput structural database
De, S., Musil, F., Ingram, T., Baldauf, C., & Ceriotti, M. (2017) · Journal of Cheminformatics · doi:10.1186/s13321-017-0192-4 · Cited by 42 · see citing works
Mansbach, R. A., & Ferguson, A. L. (2015) · The Journal of Chemical Physics · doi:10.1063/1.4914144 · Cited by 39 · see citing works
Single-Cell Computational Strategies for Lineage Reconstruction in Tissue Systems
Herring, C. A., Chen, B., McKinley, E. T., & Lau, K. S. (2018) · Cellular and Molecular Gastroenterology and Hepatology · doi:10.1016/j.jcmgh.2018.01.023 · Cited by 35 · see citing works
Machine learning in the analysis of biomolecular simulations
Kaptan, S., & Vattulainen, I. (2022) · Advances in Physics: X · doi:10.1080/23746149.2021.2006080 · Cited by 31 · see citing works
Fundamental limits on dynamic inference from single cell snapshots
Weinreb, C., Wolock, S., Tusi, B. K., Socolovsky, M., & Klein, A. M. (2017) · doi:10.1101/170118 · Cited by 27 · see citing works
Compact Time-Domain Representation for Logical Access Spoofed Audio
Karo, M., Yeredor, A., & Lapidot, I. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3341000 · Cited by 6 · see citing works
Wang, W. X., & Lefebvre, J. L. (2020) · doi:10.1101/2020.02.29.971366
- Authors
- Coifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F., & Zucker, S. W.
- Affiliations
- Yale University (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2005-05-17
- Volume
- 102
- Issue
- 21
- Pages
- 7426-7431
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC1140422
- OpenAlex
- W2006554089
- Cited by
- 1,675
- References
- 13
- Citations by year
- 2026: 69 · 2025: 97 · 2024: 101 · 2023: 120 · 2022: 118 · 2021: 119 · 2020: 113 · 2019: 107 · 2018: 88 · 2017: 93 · 2016: 71 · 2015: 79 · 2014: 85 · 2013: 66 · 2012: 73
- FWCI
- 34.70
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Neurological Disorders and Stroke (R01NS100049)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Topological and Geometric Data Analysis
- Physical Sciences › Mathematics › Applied Mathematics › Geometric Analysis and Curvature Flows
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- diffusion maps, diffusion semigroups, multiscale geometry, graph structure, transition matrix, eigenfunctions, scaling functions, manifold learning, data structure definition, harmonic analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
