Minimum description length for exponential families

A statistical model is essentially an information channel from a parameter space to a data space so that each parameter gives a distribution over possible data. We are interested in characterizing the statistical models that have finite capacity. According to the Gallager-Ryabko Theorem the capacity equals the minimax redundancy. In the minimum description length (MDL) approach to statistics one is interested in the minimax regret rather than the minimax redundancy of the statistical model. The minimax redundancy lower bounds minimax regret so if capacity is infinite the minimax regret is infinite and the MDL approach to statistics fails. In this talk we shall restrict our attention to exponential families. It has been conjectured that finite capacity implies finite minimax regret. We demonstrate that the conjecture holds in 1 dimension but is violated in 3 dimensions. This is joint work with Peter Grünwald.