jake.spiteri – Compass Blog

A post by Jake Spiteri, Compass PhD student.

Recent advancements in kernel methods have introduced a framework for nonparametric statistics by embedding and manipulating probability distributions in Hilbert spaces. In this blog post we will look at how to embed marginal and conditional distributions, and how to perform probability operations such as the sum, product, and Bayes’ rule with embeddings.

Embedding marginal distributions

Throughout this blog post we will make use of reproducing kernel Hilbert spaces (RKHS). A reproducing kernel Hilbert space is simply a Hilbert space with some additional structure, and a Hilbert space is just a topological vector space equipped with an inner product, which is also complete.

We will frequently refer to a random variable $X$ which has domain $\mathcal{X}$ endowed with the $\sigma$ -algebra $\mathcal{B}_\mathcal{X}$ .

Definition. A reproducing kernel Hilbert space $\mathcal{H}$ on $\mathcal{X}$ with associated positive semi-definite kernel function $k: \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}$ is a Hilbert space of functions $f: \mathcal{X} \rightarrow \mathbb{R}$ . The inner product $\langle\cdot,\cdot\rangle_\mathcal{H}$ satisfies the reproducing property: $\langle f, k(x, \cdot) \rangle_\mathcal{H}, \forall f \in \mathcal{H}, x \in \mathcal{X}$ . We also have $k(x, \cdot) \in \mathcal{H}, \forall x \in \mathcal{X}$ . (more…)

Author: jake.spiteri

Student Perspectives: Embedding probability distributions in RKHSs

Embedding marginal distributions