3 Gaussian distributions
3.1 Gaussian measures
3.1.1 Real Gaussian measures
The real Gaussian measure with mean \(\mu \in \mathbb {R}\) and variance \(\sigma ^2 {\gt} 0\) is the measure on \(\mathbb {R}\) with density \(\frac{1}{\sqrt{2 \pi \sigma ^2}} \exp \left(-\frac{(x - \mu )^2}{2 \sigma ^2}\right)\) with respect to the Lebesgue measure. The real Gaussian measure with mean \(\mu \in \mathbb {R}\) and variance \(0\) is the Dirac measure \(\delta _\mu \). We denote this measure by \(\mathcal{N}(\mu , \sigma ^2)\).
The characteristic function of a real Gaussian measure with mean \(\mu \) and variance \(\sigma ^2\) is given by \(x \mapsto \exp \left(i \mu x - \frac{\sigma ^2 x^2}{2}\right)\).
The central moment of order \(2n\) of a real Gaussian measure \(\mathcal{N}(\mu , \sigma ^2)\) is given by
in which \((2n - 1)!! = (2n - 1)(2n - 3) \cdots 3 \cdot 1\) is the double factorial of \(2n - 1\).
3.1.2 Gaussian measures on a Banach space
That kind of generality is not needed for this project, but we happen to have results about Gaussian measures on a Banach space in Mathlib, so we will use them. The main reference for this section is [ Hai09 ] .
Let \(F\) be a separable Banach space.
A measure \(\mu \) on \(F\) is Gaussian if for every continuous linear form \(L \in F^*\), the pushforward measure \(L_* \mu \) is a Gaussian measure on \(\mathbb {R}\).
A Gaussian measure is a probability measure.
A finite measure \(\mu \) on \(F\) is Gaussian if and only if for every continuous linear form \(L \in F^*\), the characteristic function of \(\mu \) at \(L\) is
in which \(\mathbb {V}_\mu [L]\) is the variance of \(L\) with respect to \(\mu \).
Transformations of Gaussian measures
Let \(F, G\) be two Banach spaces, let \(\mu \) be a Gaussian measure on \(F\) and let \(T : F \to G\) be a continuous linear map. Then \(T_*\mu \) is a Gaussian measure on \(G\).
Let \(\mu \) be a Gaussian measure on \(F\) and let \(c \in F\). Then the measure \(\mu \) translated by \(c\) (the map of \(\mu \) by \(x \mapsto x + c\)) is a Gaussian measure on \(F\).
The convolution of two Gaussian measures is a Gaussian measure.
Fernique’s theorem
Let \(\mu \) be a finite measure on \(F\) such that \(\mu \times \mu \) is invariant under the rotation of angle \(-\frac{\pi }{4}\). Then there exists \(C {\gt} 0\) such that the function \(x \mapsto \exp (C \Vert x \Vert ^2)\) is integrable with respect to \(\mu \).
For a Gaussian measure \(\mu \), \(\mu \times \mu \) is invariant by rotation.
For a Gaussian measure, there exists \(C {\gt} 0\) such that the function \(x \mapsto \exp (C \Vert x \Vert ^2)\) is integrable.
A Gaussian measure \(\mu \) has finite moments of all orders. In particular, there is a well defined mean \(m_\mu := \mu [\mathrm{id}]\), and for all \(L \in F^*\), \(\mu [L] = L(m_\mu )\).
A Gaussian measure has finite second moment by Lemma 3.13, hence its covariance bilinear form is well defined.
3.1.3 Gaussian measures on a finite dimensional Hilbert space
We specialize directly from Banach space to finite dimensional Hilbert space since that’s what we need in this project, although there are results for Gaussian measures on infinite dimensional Hilbert spaces that would worth stating.
A finite measure \(\mu \) on a Hilbert space \(E\) is Gaussian if and only if for every \(t \in E\), the characteristic function of \(\mu \) at \(t\) is
By Theorem 3.6, \(\mu \) is Gaussian iff for every continuous linear form \(L \in E^*\), the characteristic function of \(\mu \) at \(L\) is
Every continuous linear form \(L \in E^*\) can be written as \(L(x) = \langle t, x \rangle \) for some \(t \in E\), hence we have that \(\mu \) is Gaussian iff for every \(t \in E\),
Let \(E\) be a separable Hilbert space. We denote by \(\langle \cdot , \cdot \rangle \) the inner product on \(E\) and by \(\Vert \cdot \Vert \) the associated norm.
The characteristic function of a Gaussian measure \(\mu \) on \(E\) is given by
By Lemma 3.14, for every \(t \in E\),
By Lemma 3.13, \(\mu \) has finite first moment and \(\mu [\langle t, \cdot \rangle ] = \langle t, m_\mu \rangle \). By the same lemma, \(\mu \) has finite second moment and for any \(t\) we have \(\mathbb {V}_\mu [\langle t, \cdot \rangle ] = C'_\mu (t, t)\).
A finite measure \(\mu \) on \(E\) is Gaussian if and only if there exists \(m \in E\) and \(C\) positive semidefinite such that for all \(t \in E\), the characteristic function of \(\mu \) at \(t\) is
If that’s the case, then \(m = m_\mu \) and \(C = C'_\mu \).
Note that this lemma does not say that there exists a Gaussian measure for any such \(m\) and \(C\). We will prove that later.
Lemma 3.15 states that the characteristic function of a Gaussian measure has the wanted form.
Suppose now that there exists \(m \in E\) and \(C\) positive semidefinite such that for all \(t \in E\), \(\hat{\mu }(t) = \exp \left(i \langle t, m \rangle - \frac{1}{2} C(t, t)\right)\).
We need to show that for all \(L \in E^*\), \(L_*\mu \) is a Gaussian measure on \(\mathbb {R}\). Such an \(L\) can be written as \(\langle u, \cdot \rangle \) for some \(u \in E\). Let then \(u \in E\). We compute the characteristic function of \(\langle u, \cdot \rangle _*\mu \) at \(x \in \mathbb {R}\) with Lemma 1.5:
This is the characteristic function of a Gaussian measure on \(\mathbb {R}\) with mean \(\langle u, m \rangle \) and variance \(C(u, u)\). By Theorem 1.4, \(\langle u, \cdot \rangle _*\mu \) is Gaussian, hence \(\mu \) is Gaussian.
By Lemma 3.15, we deduce that for any \(t \in E\) we have
In particular, for any \(t\) there exists \(n_t \in \mathbb {Z}\) such that
We deduce that \(n\) is a continuous map from \(E\) to \(\mathbb {Z}\), and thus must be constant because \(E\) is connected. By looking at the value at \(t = 0\), we deduce that for any \(t\), \(n_t = 0\). Looking at real and imaginary parts we obtain that for any \(t\),
We immediately deduce that \(m = m_\mu \). Moreover, because \(C\) and \(C'_\mu \) are symmetric, they are characterized by their values on the diagonal. Indeed, for any \(x, y\),
We deduce that \(C = C'_\mu \).
Two Gaussian measures \(\mu \) and \(\nu \) on a separable Hilbert space are equal if and only if they have same mean and same covariance.
The forward direction is immediate.
For the converse direction, it is enough to show that \(\mu \) and \(\nu \) have the same characteristic function by Theorem 1.4. As they are both Gaussian, their characteristic functions only depend on their mean and covariance by Lemma 3.15. Thus they are equal.
Let \((e_1, \ldots , e_d)\) be an orthonormal basis of \(E\) and let \(\mu \) be the standard Gaussian measure on \(\mathbb {R}\). The standard Gaussian measure on \(E\) is the pushforward measure of the product measure \(\mu \times \ldots \times \mu \) by the map \(x \mapsto \sum _{i=1}^d x_i \cdot e_i\).
The fact that this definition does not depend on the choice of basis will be a consequence of the fact that its characteristic function does not depend on the basis.
For \(\mu _1, \ldots , \mu _d\) probability measures on \(\mathbb {R}\) and \(f : \mathbb {R} \to \mathbb {R}\) integrable with respect to \(\mu _i\), we have
As \(f\) is integrable, we can use Fubini theorem to obtain that
because the \(\mu _j\)s are probability measures.
The standard Gaussian measure on \(E\) is centered, i.e., \(\mu [L] = 0\) for every \(L \in E^*\).
The standard Gaussian measure is a probability measure.
The characteristic function of the standard Gaussian measure on \(E\) is given by
Denote by \(\nu \) the standard Gaussian measure on \(\mathbb {R}\). This is a straightforward computation:
The standard Gaussian measure on \(E\) is a Gaussian measure.
The mean of the standard Gaussian measure is \(0\).
The covariance matrix of the standard Gaussian measure is the identity matrix.
The multivariate Gaussian measure on \(\mathbb {R}^d\) with mean \(m \in \mathbb {R}^d\) and covariance matrix \(\Sigma \in \mathbb {R}^{d \times d}\), with \(\Sigma \) positive semidefinite, is the pushforward measure of the standard Gaussian measure on \(\mathbb {R}^d\) by the map \(x \mapsto m + \Sigma ^{1/2} x\). We denote this measure by \(\mathcal{N}(m, \Sigma )\).
The mean of the multivariate Gaussian measure \(\mathcal{N}(m, \Sigma )\) is \(m\).
The covariance matrix of the multivariate Gaussian measure \(\mathcal{N}(m, \Sigma )\) is \(\Sigma \).
A multivariate Gaussian measure is a Gaussian measure.
The characteristic function of a multivariate Gaussian measure \(\mathcal{N}(m, \Sigma )\) is given by
3.2 Gaussian processes
A process \(X : T \to \Omega \to E\) is Gaussian if for every finite subset \(t_1, \ldots , t_n \in T\), the random vector \((X_{t_1}, \ldots , X_{t_n})\) has a Gaussian distribution.
Let \(X, Y : T \to \Omega \to E\) be two stochastic processes that are modifications of each other (that is, for all \(t \in T\), \(X_t =_{a.e.} Y_t\)). If \(X\) is a Gaussian process, then \(Y\) is a Gaussian process as well.
Being a Gaussian process is defined in terms of the distribution of finite-dimensional random vectors. By Lemma 2.5, the random vector \((Y_{t_1}, \ldots , Y_{t_n})\) has the same distribution as the random vector \((X_{t_1}, \ldots , X_{t_n})\) for all \(t_1, \ldots , t_n \in T\).