Minimum description length for exponential families

  • Peter Harremoës (Copenhagen Business College, Søborg, Denmark)
University n.n. Universität Leipzig (Leipzig)


A statistical model is essentially an information channel from a parameter space to a data space so that each parameter gives a distribution over possible data. We are interested in characterizing the statistical models that have finite capacity. According to the Gallager-Ryabko Theorem the capacity equals the minimax redundancy. In the minimum description length (MDL) approach to statistics one is interested in the minimax regret rather than the minimax redundancy of the statistical model. The minimax redundancy lower bounds minimax regret so if capacity is infinite the minimax regret is infinite and the MDL approach to statistics fails. In this talk we shall restrict our attention to exponential families. It has been conjectured that finite capacity implies finite minimax regret. We demonstrate that the conjecture holds in 1 dimension but is violated in 3 dimensions. This is joint work with Peter Grünwald.

02.08.10 06.08.10

Information Geometry and its Applications III

Universität Leipzig (Leipzig) University n.n. University n.n.

Antje Vandenberg

Max-Planck-Institut für Mathematik in den Naturwissenschaften Contact via Mail

Nihat Ay

Max Planck Institute for Mathematics in the Sciences, Germany

Paolo Gibilisco

Università degli Studi di Roma "Tor Vergata", Italy

František Matúš

Academy of Sciences of the Czech Republic, Czech Republic