

Preprint 39/2010
Mixture Decomposition of Distributions using a Decomposition of the Sample Space
Guido Montúfar
Contact the author: Please use for correspondence this email.
Submission date: 01. Aug. 2010
Pages: 20
published in: Kybernetika, 49 (2013) 1, p. 23-39
Bibtex
with the following different title: Mixture decompositions of exponential families using a decomposition of their sample spaces
Download full preprint: PDF (203 kB)
Abstract:
We consider the set of join probability distributions of N binary random variables which can be written as a sum of m distributions in the following form p(x1,…,xN) = ∑ i=1mαifi(x1,…,xN), where αi ≥ 0, ∑ i=1mαi = 1, and the fi(x1,…,xN) belong to some exponential family. For our analysis we decompose the sample space into portions on which the mixture components fi can be chosen arbitrarily. We derive lower bounds on the number of mixture components from a given exponential family necessary to represent distributions with arbitrary correlations up to a certain order or to represent any distribution. For instance, in the case where fi are independent distributions we show that every distribution p on {0,1}N is contained in the mixture model whenever m ≥ 2N-1, and furthermore, that there are distributions which are not contained in the mixture model whenever m < 2N-1.