The DOI matches its Crossref record; 234 of the 234 references it cites were found in a registry or an index.
Kernel Mean Embedding of Distributions: A Review and Beyond
Open accessAbstract
A Hilbert space embedding of a distribution—in short, a kernel mean embedding—has recently emerged as a powerful tool for machine learning and statistical inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original “feature map” common to support vector machines (SVMs) and other kernel methods. In addition to the classical applications of kernel methods, the kernel mean embedding has found novel applications in fields ranging from probabilistic modeling to statistical inference, causal discovery, and deep learning. This survey aims to give a comprehensive review of existing work and recent advances in this research area, and to discuss challenging issues and open problems that could potentially lead to new research directions. The survey begins with a brief introduction to the RKHS and positive definite kernels which forms the backbone of this survey, followed by a thorough discussion of the Hilbert space embedding of marginal distributions, theoretical guarantees, and a review of its applications. The embedding of distributions enables us to apply RKHS methods to probability measures which prompts a wide range of applications such as kernel two-sample testing, independent testing, and learning on distributional data. Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications. The conditional mean embedding enables us to perform sum, product, and Bayes' rules—which are ubiquitous in graphical model, probabilistic inference, and reinforcement learning— in a non-parametric way using this new representation of distributions. We then discuss relationships between this framework and other related areas. Lastly, we give some suggestions on future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–234 of 234
Who Supported Obama in 2012? Ecological Inference through Distribution Regression
Flaxman, S. R., Wang, Y.-X., & Smola, A. J. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783300 · Cited by 45
Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential FamiliesPreprint
Strathmann, H., Sejdinović, D., Livingstone, S., Szabó, Z., & Gretton, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02564 · Cited by 42
A Kernel Independence Test for Random ProcessesPreprint
Chwialkowski, K., & Gretton, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.4501 · Cited by 42
DISCO Nets: DISsimilarity COefficient NetworksPreprint
Bouchacourt, D., Mudigonda, P. K., & Nowozin, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02556 · Cited by 42
On the optimal estimation of probability measures in weak and strong topologies
Sriperumbudur, B. K. (2016) · Bernoulli · doi:10.3150/15-bej713 · Cited by 41
Self-citationRamdas, A., Reddi, S. J., Póczos, B., Singh, A., & Wasserman, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.00655 · Cited by 41
Causal Discovery via Reproducing Kernel Hilbert Space Embeddings
Chen, Z., Zhang, K., Chan, L., & Schölkopf, B. (2014) · Neural Computation · doi:10.1162/neco_a_00599 · Cited by 39
Self-citationHilbert Space Embeddings of POMDPsPreprint
Nishiyama, Y., Boularias, A., Gretton, A., & Fukumizu, K. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.4887 · Cited by 37
Self-citationKernel Mean Shrinkage EstimatorsPreprint
Muandet, K., Sriperumbudur, B. K., Fukumizu, K., Gretton, A., & Schölkopf, B. (2014) · Journal of Machine Learning Research · doi:10.48550/arxiv.1405.5505 · Cited by 34
Self-citationComputing functions of random variables via reproducing kernel Hilbert space representations
Schölkopf, B., Muandet, K., Fukumizu, K., Stefan Harmeling, B., & Peters, J. (2015) · Statistics and Computing · doi:10.1007/s11222-015-9558-5 · Cited by 32
Self-citationGaussian Processes for Independence Tests with Non-iid Data in Causal Inference
Flaxman, S. R., Neill, D. B., & Smola, A. J. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2806892 · Cited by 32
Cooperative sensing and recognition by a swarm of mobile robots
Giusti, A., Nagi, J., Gambardella, L. M., et al. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385982 · Cited by 31
Nonparametric kernel estimators for image classification
Póczos, B., Xiong, L., Sutherland, D. J., et al. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248028 · Cited by 30
Decentralization of Multiagent Policies by Learning What to Communicate
Paulos, J., Chen, S. W., Shishika, D., & Kumar, V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793777 · Cited by 28
A Kernel Test for Three-Variable InteractionsPreprint
Sejdinović, D., Gretton, A., & Bergsma, W. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.2281 · Cited by 27
Herding Dynamic Weights for Partially Observed Random Field ModelsPreprint
Welling, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2605 · Cited by 26
Kernel Embeddings of Latent Tree Graphical Models
Song, L., P. Parikh, A., & P Xing, E. (2018) · Figshare · doi:10.1184/r1/6475826 · Cited by 26
A Wild Bootstrap for Degenerate Kernel TestsPreprint
Chwialkowski, K., Sejdinović, D., & Gretton, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5404 · Cited by 26
Hypothesis testing using pairwise distances and associated kernelsPreprint
Sejdinović, D., Gretton, A., Fukumizu, K., & Sriperumbudur, B. K. (2012) · arXiv (Cornell University) · Cited by 26
Self-citationA Framework for Probability Density Estimation
Shawe-Taylor, J. S., & Dolia, A. N. (2007) · Cited by 24
Latent Support Measure Machines for Bag-of-Words Data Classification
Yoshikawa, Y., Iwata, T., & Sawada, H. (2014) · neural information processing systems · Cited by 24
Learning in Hilbert vs. Banach Spaces: A Measure Embedding Viewpoint
Fukumizu, K., Lanckriet, G. R. G., & Sriperumbudur, B. K. (2011) · neural information processing systems · Cited by 23
Self-citationOn the High Dimensional Power of a Linear-Time Two Sample Test under Mean-shift Alternatives
Reddi, S. J., Ramdas, A., Póczos, B., Singh, A., & Wasserman, L. (2015) · International Conference on Artificial Intelligence and Statistics · Cited by 22
Otte, M. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918779704 · Cited by 21
Statistical inference using weak chaos and infinite memory
Welling, M., & Chen, Y. (2010) · Journal of Physics Conference Series · doi:10.1088/1742-6596/233/1/012005 · Cited by 19
Filtering with State-Observation Examples via Kernel Monte Carlo Filter
Kanagawa, M., Nishiyama, Y., Gretton, A., & Fukumizu, K. (2015) · Neural Computation · doi:10.1162/neco_a_00806 · Cited by 17
Self-citationOn the Schoenberg Transformations in Data Analysis: Theory and Illustrations
Bavaud, F. (2011) · Journal of Classification · doi:10.1007/s00357-011-9092-x · Cited by 17
Mixture density estimation via Hilbert space embedding of measures
Sriperumbudur, B. K. (2011) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2011.6033685 · Cited by 11
Self-citationJust-In-Time Learning for Fast and Flexible Inference
Eslami, S. M. A., Tarlow, D., Kohli, P., & Winn, J. M. (2014) · neural information processing systems · Cited by 11
Minimax Estimation of Kernel Mean EmbeddingsPreprint
Tolstikhin, I., Sriperumbudur, B. K., & Muandet, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04361 · Cited by 10
Self-citationA low variance consistent test of relative dependencyPreprint
Bounliphone, W., Gretton, A., Tenenhaus, A., & Blaschko, M. B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.3852 · Cited by 9
Kernel Mean Estimation via Spectral FilteringPreprint
Muandet, K., Sriperumbudur, B. K., & Schölkopf, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.0900 · Cited by 9
Self-citationHarmeling, S., Hirsch, M., & Schölkopf, B. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.144 · Cited by 3
Self-citationChen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963
The Born supremacy: quantum advantage and training of an Ising Born machine
Coyle, B., Mills, D., Danos, V., & Kashefi, E. (2020) · npj Quantum Information · doi:10.1038/s41534-020-00288-9 · Cited by 155
In Search of Lost Domain Generalization
Gulrajani, I., & Lopez-Paz, D. (2020) · arXiv · doi:10.48550/arxiv.2007.01434 · Cited by 440
Metadata
- Authors
- Muandet, K., Fukumizu, K., Sriperumbudur, B., & Schölkopf, B.
- Venue
- Foundations and Trends® in Machine Learning
- Publisher
- Emerald
- Published
- 2017
- Volume
- 10
- Issue
- 1-2
- Pages
- 1-141
- ISSN
- 1935-8237 (ISSN-L)
- 1935-8245
- OpenAlex
- W2418306335
- Cited by
- 409
- References
- 234
- FWCI
- 15.42
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
