Delve into the future of research at MiS with our preprint repository. Our scientists are making groundbreaking discoveries and sharing their latest findings before they are published. Explore repository to stay up-to-date on the newest developments and breakthroughs.
MiS Preprint
39/2010
Mixture Decomposition of Distributions using a Decomposition of the Sample Space
Guido Montúfar
Abstract
We consider the set of join probability distributions of $N$ binary random variables which can be written as a sum of $m$ distributions in the following form $p(x_1,\ldots,x_N)=\sum_{i=1}^m \alpha_i f_i(x_1,\ldots,x_N)$, where $\alpha_i \geq 0$, $\sum_{i=1}^m \alpha_i =1$, and the $f_i(x_1,\ldots,x_N)$ belong to some exponential family.
For our analysis we decompose the sample space into portions on which the mixture components $f_i$ can be chosen arbitrarily. We derive lower bounds on the number of mixture components from a given exponential family necessary to represent distributions with arbitrary correlations up to a certain order or to represent any distribution.
For instance, in the case where $f_i$ are independent distributions we show that every distribution $p$ on $\{0,1\}^N$ is contained in the mixture model whenever $m\geq 2^{N-1}$, and furthermore, that there are distributions which are not contained in the mixture model whenever $m<2^{N-1}$.
