New links between information and geometry

This talk will summarize two recent works at the crossroads of information theory and geometry.

The first one deals with the “typical realizations” of a memoryless random source whose law is a stratified measure: a convex combination of rectifiable measures. Stratified measures generalize discrete-continuous mixtures and may have a singular continuous part (which is “carried” by manifolds in various dimensions). We state an asymptotic equipartition property for stratified measures that shows concentration of probability on subsets of a few "typical dimensions" and that quantifies the volume growth of typical sequences in each stratum. This gives a concrete interpretation for Renyi’s information dimension in some cases.

The second concerns the magnitude of (possibly enriched) categories, which gives, in particular, a new isometric invariant of metric spaces. Magnitude can be seen as a categorical generalization of cardinality and shares many properties with it. In turn, the Boltzmann-Shannon entropy is a probabilistic extension of (the logarithm of) cardinality. We aim to connect the two ideas by considering a generalization of Shannon entropy to finite categories that also serves as a probabilistic extension of magnitude.