Mixture Decomposition of Distributions using a Decomposition of the Sample Space

Guido Montúfar

Contact the author: Please use for correspondence this email.
Submission date: 01. Aug. 2010
Pages: 20
published in: Kybernetika, 49 (2013) 1, p. 23-39 
Bibtex
with the following different title: Mixture decompositions of exponential families using a decomposition of their sample spaces
Download full preprint: PDF (203 kB)

Abstract:

We consider the set of join probability distributions of N binary random variables which can be written as a sum of m distributions in the following form p(x₁,…,x_N) = ∑ _i=1^mα_if_i(x₁,…,x_N), where α_i ≥ 0, ∑ _i=1^mα_i = 1, and the f_i(x₁,…,x_N) belong to some exponential family. For our analysis we decompose the sample space into portions on which the mixture components f_i can be chosen arbitrarily. We derive lower bounds on the number of mixture components from a given exponential family necessary to represent distributions with arbitrary correlations up to a certain order or to represent any distribution. For instance, in the case where f_i are independent distributions we show that every distribution p on {0,1}^N is contained in the mixture model whenever m ≥ 2^N-1, and furthermore, that there are distributions which are not contained in the mixture model whenever m < 2^N-1.