Optimally approximating exponential families

Johannes Rauh

Contact the author: Please use for correspondence this email.
Submission date: 28. Oct. 2011
published in: Kybernetika, 49 (2013) 2, p. 199-215 
Bibtex
MSC-Numbers: 62E17, 94A17, 60E05
Keywords and phrases: exponential family, information divergence, hierarchical models
Download full preprint: PDF (212 kB), PS ziped (321 kB)

Abstract:
This article studies exponential families on finite sets such that the information divergence D(P∥) of an arbitrary probability distribution from is bounded by some constant D > 0. A particular class of low-dimensional exponential families that have low values of D can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. Exponential families where D = log(2) are studied in detail. This case is special, because if D < log(2), then contains all probability measures with full support.