deFinetti’s Theorem Part II
Some further references on deFinetti’s Theorem
This paper shows how the problem of exchangeability can be interpreted geometrically and proves some “approximate” versions of the theorem for finite sequences.
- G. R. Wood, Binomial Mixtures and Finite Exchangeability.
This paper considers the problem of extending a finite exchangeable sequence to an infinite one; and also the problem of determining how likely it is that a randomly chosen distribution on $0,\ldots, n$ will be a mixture of binomial distributions. The questions turn out to be closely related. The paper gives a formula computing the fraction of binomial mixtures among all distributions (which turns out to go to zero VERY quickly with increasing $n$) and then relates that to the probability that an $n$ exchangeable sequence is infinitely extendible (which also goes to zero very quickly with $n$).
The results amount to some (non-trivial!) calculations of volumes of regions in simplices.
Apparently as of the time of this paper there was a conjecture due to Crisma giving a formula for the probability that an exchangeable sequence of $n$ random variables can be extended to one of length $r$ that was unsolved, what is its current status?
- P. Diaconis and D. Freedman, Finite Exchangeable Sequences
This paper computes the distance between the distribution of $k$ exchangeable random variables taking values in a (finite) set $S$ and the closest mixture of IID random variables. To explain (one of) their theorems, represent a probability distribution on $S$ as a point in the $|S|$-dimensional simplex. Suppose $\mu$ is a probability distribution on this simplex. Given such a probability distribution $\mu$, let $P(k,\mu)$ be the distribution on $S$ from $\mu$, and then making $k$ iid choices from this $\mu$.
Theorem: Let $S$ be a finite set with $|S|$ elements. Let $P$ be an exchangeable probability on $S^{n}$. Then there is a probability $\mu$ on the $|S|$-simplex so that $| P_{k}-P(k,\mu ) | \le 2|S| k/n$ for all $k\le n$. Here $P_{k}$ is the marginal probability of $P$ on sequences of length $k\le n$.
In other words, if $P$ is a distribution on sequences of length $k$ that can be extended to an exchangeable sequence of length $n$, then $P$ is within distance $k/n$ of a “mixture.”
The distance here is “variation distance” $|P-Q|=2\sup_A|P(A)-Q(A)|$ over Borel sets $A$.