Preprint 73/2011

Optimally approximating exponential families

Johannes Rauh

Contact the author: Please use for correspondence this email.
Submission date: 28. Oct. 2011
published in: Kybernetika, 49 (2013) 2, p. 199-215 
MSC-Numbers: 62E17, 94A17, 60E05
Keywords and phrases: exponential family, information divergence, hierarchical models
Download full preprint: PDF (212 kB), PS ziped (321 kB)

This article studies exponential families E on finite sets such that the information divergence D(PE) of an arbitrary probability distribution from E is bounded by some constant D > 0. A particular class of low-dimensional exponential families that have low values of D can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. Exponential families where D = log(2) are studied in detail. This case is special, because if D < log(2), then E contains all probability measures with full support.

26.04.2023, 02:16