An Exchangeable Law of Large Numbers

In the proof of De Finetti’s Theorem in my last post, I got to a section where I needed to show a particular convergence property of a sequence of exchangeable random variables. For independent identically distributed RVs, we have Kolmogorov’s 0-1 law, and in particular a strong law of large numbers. Does a version of this result hold for exchangeable sequences? As these represent only a mild generalisation of iid sequences, we might hope so. The following argument demonstrates that this is true, as well as providing a natural general proof of De Finetti.

Define $\mathcal{E}_n=\sigma(\{f(X_1,\ldots,X_n): f\text{ symmetric, Borel}\})$ , the smallest sigma-field wrt which the first n RVs are exchangeable. Note that $\mathcal{E}_1\supset\mathcal{E}_2\supset\ldots\supset \mathcal{E}=\cap_n\mathcal{E}_n$ , the exchangeable sigma-field.

So now take g(X) symmetric in the first n variables. By exchangeability $E[\frac{1}{n}\sum_1^n f(X_j)g(X)]=E[f(X_1)g(X)]$ . Now set $g=1_A$ , for $A\in\mathcal{E}_n$ , and so because the LHS integrand is $\mathcal{E}_n$ -meas. we have $Z_n=\frac{1}{n}\sum_1^n f(X_j)=E[f(X_1)|\mathcal{E}_n]$ . So Z is a backwards martingale.

We have a convergence theorem for backwards martingales, which tells us that $\lim_n n^{-1}\sum^n f(X_j)$ exists, and in fact $= E[f(X_1)|\mathcal{E}]$ almost surely. Setting $f(X)=1(X\leq x)$ gives that $\lim_n\frac{\#\{X_i\leq x: i\leq n\}}{n}=F(x):=P(X_1\leq x|\mathcal{E})$ . We now perform a similar procedure for functions defined on the first k RVs, in an attempt to demonstrate independence.

For $f:\mathbb{R}^k\rightarrow\mathbb{R}$ , we seek a backwards martingale, so we take sums over the $n^{(k)}$ ways to choose k of the first n RVs. So $\frac{1}{n(n-1)\ldots(n-k+1)}\sum_{I\subset[n]} f(X_{i_1},\ldots,X_{i_k})$ is a backwards martingale, and hence $E[f(X_1,\ldots,X_k)|\mathcal{E}]=\lim_n \frac{1}{n(n-1)\ldots(n-k+1)}\sum f(-)$ . As before, set $f(y_1,\ldots,y_k)=1(y_1\leq x_1)\ldots 1(y_k\leq x_k)$ . Crucially, we can replace the falling factorial term with $n^{-k}$ as we are only considering the limit, then exchange summation as everything is positive and nice to get: $E[f(X_1,\ldots,X_k)|\mathcal{E}]=\lim(\frac{1}{n}\sum 1(X_1\leq x_1))\ldots(\frac{1}{n}\sum 1(X_k\leq x_k))$ thus demonstrating independence of $(X_n)$ conditional on $\mathcal{E}$ .

So what have we done? Well, we’ve certainly proven de Finetti in the most general case, and we have in addition demonstrated the existence of a Strong Law of Large Numbers for exchangeable sequences, where the limit variable is $\mathcal{E}$ -measurable.

Eventually Almost Everywhere

A blog about probability and olympiads by Dominic Yeo

An Exchangeable Law of Large Numbers

Leave a Reply Cancel reply

Share this:

Like this:

Related

Leave a Reply Cancel reply