A post by Jake Spiteri, Compass PhD student.
Recent advancements in kernel methods have introduced a framework for nonparametric statistics by embedding and manipulating probability distributions in Hilbert spaces. In this blog post we will look at how to embed marginal and conditional distributions, and how to perform probability operations such as the sum, product, and Bayes’ rule with embeddings.
Embedding marginal distributions
Throughout this blog post we will make use of reproducing kernel Hilbert spaces (RKHS). A reproducing kernel Hilbert space is simply a Hilbert space with some additional structure, and a Hilbert space is just a topological vector space equipped with an inner product, which is also complete.
We will frequently refer to a random variable which has domain endowed with the -algebra .
Definition. A reproducing kernel Hilbert space on with associated positive semi-definite kernel function is a Hilbert space of functions . The inner product satisfies the reproducing property: . We also have . (more…)