6 The central limit theorem

6.1 The central limit theorem for real random variables

Lemma 6.1

✓

Let \(X\) be a real random variable with characteristic function \(\phi \) with \(\mathbb {E}[\vert X \vert ^n] {\lt} \infty \) and let \(k \le n\). Then the \(k\)^th derivative of \(\phi \) is \(\phi ^{(k)}(t) = \mathbb {E}[i X]^k e^{i t X}\)   .

Proof ▶

Lemma 6.2 Peano form of Taylor’s theorem

✓

For a \(n\)-times continuously differentiable function \(f : \mathbb {R} \to E\), for any \(x_0\in \mathbb {R}\),

\begin{align*} f(x) - \sum _{i=0}^n\frac{f^{(i)}(x_0)}{i!}(x-x_0)^i = o((x - x_0)^k) \: . \end{align*}

Proof ▶

Lemma 6.3

✓

A finite collection of random variables are independent iff their joint law is the product of their respective laws.

Proof ▶

Lemma 6.4

✓

Let \(X\) be a real random variable with characteristic function \(\phi \), with \(\mathbb {E}[\vert X \vert ^n] {\lt} \infty \). Then as \(t \to 0\),

\begin{align*} \phi (t) = \sum _{k=0}^n \frac{(it)^k \mathbb {E}[X^k]}{k!} + o(t^n) \: . \end{align*}

Proof ▶

Lemma 6.5

✓

For \(t\in \mathbb {C}\), \(\lim _{n\to \infty }(1+t/n+o(1/n))^n=\exp (t)\) (where the little-\(o\) term may be complex).

Proof ▶

Theorem 6.6 Central limit theorem

✓

Let \(X_1, X_2, \ldots \) be i.i.d. real random variables with mean 0 and variance 1, and let \(Z\) be a random variable with law \(\mathcal N(0,1)\). Then

\begin{align*} \frac{1}{\sqrt{n}}\sum _{k=1}^n X_k \xrightarrow {d} Z \: . \end{align*}

Proof ▶

Let \(S_n = \frac{1}{\sqrt{n}}\sum _{k=1}^n X_k\). Let \(\phi \) be the characteristic function of \(X_k\). By Lemma 4.6 and 4.7, the characteristic function of \(S_n\) is \(\phi _n(t) = (\phi (n^{-1/2}t))^n\).

By Lemma 6.4,

\begin{align*} \phi _n(t) = (\phi (n^{-1/2}t))^n = \left(1 - \frac{1}{2n}t^2 + o(\frac{1}{n})\right)^n \to _{n \to +\infty } e^{-t^2/2} \: . \end{align*}

Since the r.h.s. is the characteristic function of \(Z\), we conclude that \(\frac{1}{\sqrt{n}}\sum _{k=1}^n X_k \xrightarrow {d} Z\) by Lemma 4.15.

6.2 The central limit theorem for random variables in \(\mathbb {R}^d\)

Theorem 6.7 Cramér-Wold

Let \(X, X_1, X_2, \ldots \) be random variables in \(\mathbb {R}^d\). Then \(X_n \xrightarrow {d} X\) iff for every \(a \in \mathbb {R}^d\), \(\langle a, X_n \rangle \xrightarrow {d} \langle a, X \rangle \).

Proof ▶

By Theorem 4.15, the convergence in distribution of the random variables is equivalent to the convergence of their characteristic functions for each \(a \in \mathbb {R}^d\), which for each such \(a\) is equivalent to the convergence in distribution of the random variables \(\langle a, X_n \rangle \) by the same theorem.

Theorem 6.8 Central limit theorem in \(\mathbb {R}^d\)

Let \(X_1, X_2, \ldots \) be i.i.d. random variables in \(\mathbb {R}^d\) with mean 0 and identity covariance matrix, and let \(Z\) be a random variable with law \(\mathcal N(0,I_d)\). Then

\begin{align*} \frac{1}{\sqrt{n}}\sum _{k=1}^n X_k \xrightarrow {d} Z \: . \end{align*}

Proof ▶