Data Fusion and Multicue Data Matching by Diffusion Maps
Abstract
Data fusion and multicue data matching are fundamental tasks of high-dimensional data analysis. In this paper, we apply the recently introduced diffusion framework to address these tasks. Our contribution is three-fold: First, we present the Laplace-Beltrami approach for computing density invariant embeddings which are essential for integrating different sources of data. Second, we describe a refinement of the Nyström extension algorithm called "geometric harmonics." We also explain how to use this tool for data assimilation. Finally, we introduce a multicue data matching scheme based on nonlinear spectral graphs alignment. The effectiveness of the presented schemes is validated by applying it to the problems of lipreading and image sequence alignment.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Diffusion maps transform data from high dimensional gene expression space to a space of much lower dimensions, where usual Euclidean distance is reflective of the cellular phenotypic distance.
“Essentially, they transform data from high dimensional gene expression space to a space of much lower dimensions, where usual Euclidean distance is reflective of the cellular phenotypic distance.”
From Occult polyclonality of preclinical pancreatic cancer models drives in vitro evolution (2022) · cites this paper as [62]
Cites
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,842 · see citing works
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · Cited by 15,411 · see citing works
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Belkin, M., & Niyogi, P. (2003) · Neural Computation · doi:10.1162/089976603321780317 · Cited by 7,562 · see citing works
Chung, F. (1996) · Regional conference series in mathematics · doi:10.1090/cbms/092 · Cited by 5,918 · see citing works
Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering
Belkin, M., & Niyogi, P. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0080 · Cited by 4,534 · see citing works
Coifman, R. R., & Lafon, S. (2006) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2006.04.006 · Cited by 2,380 · see citing works
Self-citationModern Multidimensional Scaling
Borg, I., & Groenen, P. J. F. (1997) · Springer series in statistics · doi:10.1007/978-1-4757-2711-1 · Cited by 1,745 · see citing works
Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data
Donoho, D. L., & Grimes, C. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1031596100 · Cited by 1,704 · see citing works
Coifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F., & Zucker, S. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500334102 · Cited by 1,675 · see citing works
Self-citationModern multidimensional scaling : theory and applications
Borg, I., & Groenen, P. J. F. (1997) · Springer eBooks · doi:10.1007/0-387-28981-x · Cited by 1,615 · see citing works
Spectral grouping using the nystrom method
Fowlkes, C. C., Belongie, S., Chung, F., & Malik, J. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262185 · Cited by 1,310 · see citing works
Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
Nadler, B., Lafon, S., Coifman, R. R., & Kevrekidis, I. G. (2006) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2005.07.004 · Cited by 989 · see citing works
Self-citationMulticlass spectral clustering
Yu, & Shi (2003) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238361 · Cited by 983 · see citing works
Robust registration of 2D and 3D point sets
Fitzgibbon, A. W. (2003) · Image and Vision Computing · doi:10.1016/j.imavis.2003.09.004 · Cited by 925 · see citing works
Geometric Bounds for Eigenvalues of Markov Chains
Diaconis, P., & Stroock, D. (1991) · The Annals of Applied Probability · doi:10.1214/aoap/1177005980 · Cited by 882 · see citing works
Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering
Bengio, Y., Paiement, J.-F., Vincent, P., et al. (2003) · Cited by 864 · see citing works
Segmentation using eigenvectors: a unifying view
Weiss, Y. (1999) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1999.790354 · Cited by 729 · see citing works
Diffusion Kernels on Graphs and Other Discrete Input Spaces
Kondor, R., & Lafferty, J. (2002) · Cited by 728 · see citing works
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669 · see citing works
A Random Walks View of Spectral Segmentation.
Meilă, M., & Shi, J. (2001) · Cited by 628 · see citing works
Lafon, S., & Lee, A. B. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.184 · Cited by 620 · see citing works
Self-citationGeometric hashing: an overview
Wolfson, H. J., & Rigoutsos, I. (1997) · IEEE Computational Science and Engineering · doi:10.1109/99.641604 · Cited by 562 · see citing works
Extraction of visual features for lipreading
Matthews, I. A., Cootes, T. F., Bangham, J., Cox, S. J., & Harvey, R. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982900 · Cited by 538 · see citing works
Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space AlignmentPreprint
Zhang, Z., & Zha, H. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212008 · Cited by 536 · see citing works
Dynamic Bayesian Networks for Audio-Visual Speech Recognition
Nefian, A., Liang, L., Pi, X., Liu, X., & Murphy, K. (2002) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865702206083 · Cited by 320 · see citing works
Semisupervised alignment of manifolds.
Ham, J., Lee, D. D., & Saul, L. K. (2005) · International Conference on Artificial Intelligence and Statistics · Cited by 254 · see citing works
Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions
Coifman, R. R., & Lafon, S. (2006) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2005.07.005 · Cited by 230 · see citing works
Self-citationCoifman, R. R., Lafon, S., Lee, A. B., Maggioni, M., Nadler, B., Warner, F. J., & Zucker, S. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500896102 · Cited by 217 · see citing works
Self-citationExact and approximate graph matching using random walks
Gori, M., Maggini, M., & Sarti, L. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.138 · Cited by 160 · see citing works
Towards a Theoretical Foundation for Laplacian-Based Manifold Methods
Belkin, M. A., & Niyogi, P. (2005) · Lecture notes in computer science · doi:10.1007/11503415_33 · Cited by 151 · see citing works
Robust Registration of 2D and 3D Point Sets
Fitzgibbon, A. W. (2001) · British Machine Vision Conference (BMVC) · doi:10.5244/c.15.43 · Cited by 148 · see citing works
Photo-realistic talking-heads from image samples
Cosatto, E., & Graf, H. P. (2000) · IEEE Transactions on Multimedia · doi:10.1109/6046.865480 · Cited by 125 · see citing works
Speechreading using Probabilistic Models
Luettin, J., & Thacker, N. A. (1997) · Computer Vision and Image Understanding · doi:10.1006/cviu.1996.0570 · Cited by 123 · see citing works
Improving connected letter recognition by lipreading
Bregler, C., Hild, H., Manke, S., & Waibel, A. H. (1993) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1993.319179 · Cited by 123 · see citing works
Active Shape Models for Visual Speech Feature Extraction
Luettin, J., Thacker, N. A., & Beet, S. W. (1996) · NATO ASI series. Series F : Computer and system sciences · doi:10.1007/978-3-662-13015-5_28 · Cited by 84 · see citing works
Video Rewrite: Driving Visual Speech with Audio
Bregler, C., Covell, M., & Slaney, M. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596787 · Cited by 67 · see citing works
Many-to-many graph matching via metric embedding
Keselman, Y., Shokoufandeh, A., Demirci, M. F., et al. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211441 · Cited by 49 · see citing works
Robust Lip Tracking by Combining Shape, Color and Motion
Tian, Y., Kanade, T., & Cohn, J. F. (2000) · Cited by 40 · see citing works
Visually Recognizing Speech Using Eigensequences
Li, N., Dettmer, S., & Shah, M. A. (1997) · Computational imaging and vision · doi:10.1007/978-94-015-8935-2_15 · Cited by 24 · see citing works
Representation Analysis and Synthesis of Lip Images Using Dimensionality Reduction
Aharon, M., & Kimmel, R. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-5166-3 · Cited by 21 · see citing works
Probabilistic Models of Verbal and Body Gestures
Bregler, C., Omohundro, S., Covell, M., et al. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511569937.016 · Cited by 11 · see citing works
Graph matching using spectral embedding and alignment
Bai, X. P., Yu, H., & Hancock, E. R. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1334550 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Rapid fluctuations in functional connectivity of cortical networks encode spontaneous behavior
Benisty, H., Barson, D., Moberly, A. H., Lohani, S., Tang, L., Coifman, R. R., Crair, M. C., Mishne, G., Cardin, J. A., & Higley, M. J. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01498-y · Cited by 65 · see citing works
Self-citationOccult polyclonality of preclinical pancreatic cancer models drives in vitro evolution
Monberg, M. E., Geiger, H., Lee, J. J., Sharma, R., Semaan, A., Bernard, V., Wong, J., Wang, F., Liang, S., Swartzlander, D. B., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-31376-3 · Cited by 30 · see citing works
Compact Time-Domain Representation for Logical Access Spoofed Audio
Karo, M., Yeredor, A., & Lapidot, I. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3341000 · Cited by 6 · see citing works
- Authors
- Lafon, S., Keller, Y., & Coifman, R. R.
- Affiliations
- Google (United States) (US)
- Yale University (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2006-11
- Volume
- 28
- Issue
- 11
- Pages
- 1784-1797
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2108759471
- Cited by
- 312
- References
- 43
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 4 · 2023: 14 · 2022: 11 · 2021: 13 · 2020: 19 · 2019: 15 · 2018: 15 · 2017: 20 · 2016: 21 · 2015: 21 · 2014: 21 · 2013: 24 · 2012: 17
- FWCI
- 18.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Topological and Geometric Data Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Mathematics › Applied Mathematics › Mathematical Analysis and Transform Methods
- Keywords
- diffusion maps, data fusion, Nyström extension, data assimilation, lip reading
- MeSH terms
- Algorithms, Artificial Intelligence, Image Enhancement, Image Interpretation, Computer-Assisted, Pattern Recognition, Automated, Subtraction Technique, Cluster Analysis, Databases, Factual, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review