Research Topic
Exponential Families & Information Maximization
Exponential families are natural statistical models. In physics they are used since their elements maximize the entropy subject to constrained expectation values of a fixed set of associated observables. An important subclass are the graphical and hierarchical (log linear) models that are used to model interactions between different random variables. They also appear in information geometry and algebraic statistics due to their nice structural properties.
The information distance from an exponential family has an interpretation as information loss through a projection onto that family. Mutual information, conditional mutual information and multi-information allow for such a geometric interpretation. In this project we analyze the maximization of the distance from exponential families. This problem is motivated by principles of information maximization known from theoretical neuroscience. The project aims at identifying natural models of learning systems that are consistent with information maximization and, at the same time, display high generalization ability. In this context, topological closures of exponential families turn out to be essential. Geometrically they are equivalent to polytopes and display a rich combinatorial structure.
inJournal
2019
Journal Open Access
Canonical divergence for measuring classical and quantum complexity
In: Entropy, 21 (2019) 4, p. 435inBook
2018
Repository Open Access
Comparison and connection between the joint and the conditional generalized iterative scaling algorithm
In: Proceedings of the 11th workshop on uncertainty processing WUPES '18, June 6-9, 2018 / Václav Kratochvíl (ed.)Praha : MatfyzPress, 2018. - pp. 105-116
inBook
2015
Standard divergence in manifold of dual affine connections
In: Geometric science of information : second international conference, GSI 2015, Palaiseau, France, October 28-30, 2015, proceedings / Frank Nielsen... (eds.)Cham : Springer, 2015. - pp. 320-325
(Lecture notes in computer science ; 9389)
inJournal
2014
Repository Open Access
Positive margins and primary decomposition
In: Journal of commutative algebra, 6 (2014) 2, pp. 173-208inJournal
2014
Journal Open Access
Scaling of model approximation errors and expected entropy distances
In: Kybernetika, 50 (2014) 2, pp. 234-245inBook
2013
Repository Open Access
Maximal information divergence from statistical models defined by neural networks
In: Geometric science of information : first international conference, GSI 2013, Paris, France, August 28-30, 2013. Proceedings / Frank Nielsen... (eds.)Berlin [u. a.] : Springer, 2013. - pp. 759-766
(Lecture notes in computer science ; 8085)
inJournal
2013
Journal Open Access
Optimally approximating exponential families
In: Kybernetika, 49 (2013) 2, pp. 199-215inJournal
2011
Repository Open Access
Finding the maximizers of the information divergence from an exponential family
In: IEEE transactions on information theory, 57 (2011) 6, pp. 3236-3247Academic
2011
Repository Open Access
Finding the maximizers of the information divergence from an exponential family
Dissertation, Universität Leipzig, 2011inBook
2011
Maximization of the information divergence from an exponential family and criticality
In: IEEE international symposium on information theory proceedings (ISIT) 2011 : July 31-August 5, 2011 in St. Petersburg, RussiaPiscataway, NY : IEEE, 2011. - pp. 903-907
inJournal
2011
Repository Open Access
Support sets in exponential families and oriented matroid theory
In: International journal of approximate reasoning, 52 (2011) 5, pp. 613-626inJournal
2010
Journal Open Access
Neighborliness of marginal polytopes
In: Beiträge zur Algebra und Geometrie, 51 (2010) 1, pp. 45-56inJournal
2009
Journal Open Access
Hierarchical models, marginal polytopes, and linear codes
In: Kybernetika, 45 (2009) 2, pp. 189-207inJournal
2007
High-resolution multiple-unit EEG in cat auditory cortex reveals large spatio-temporal stochastic interactions
In: Biosystems, 89 (2007) 1/3, pp. 190-197inJournal
2006
A temporal learning rule in recurrent systems supports high spatio-temporal stochastic interactions
In: Neurocomputing, 69 (2006) 10/12, pp. 1199-1202inJournal
2006
Journal Open Access
Maximizing multi-information
In: Kybernetika, 42 (2006) 5, pp. 517-538inBook
2006
Repository Open Access
Support sets of distributions with given interaction structure
In: 7th Workshop on Uncertainty Processing : WUPES'06 ; Mikulov, Czech Republik ; 16-20th September 2006Praha : Academy of Sciences of the Czech Republik / Institute of Information Theory and Automation, 2006. - pp. 52-61
inJournal
2005
Finite state automata resulting from temporal information maximization and a temporal learning rule
In: Neural computation, 17 (2005) 10, pp. 2258-2290inJournal
2005
Stochastic interaction in associative nets
In: Neurocomputing, 65 (2005), pp. 387-392inJournal
2003
Repository Open Access
Dynamical properties of strongly interacting Markov chains
In: Neural networks, 16 (2003) 10, pp. 1483-1497inBook
2003
Repository Open Access
On maximization of the information divergence from an exponential family
In: Proceedings of 6th workshop on uncertainty processing : Hejnice, September 24-27, 2003[Praha] : Oeconomica, 2003. - pp. 199-204
inJournal
2003
Spatial and temporal stochastic interaction in neuronal assemblies
In: Theory in biosciences, 122 (2003) 1, pp. 5-18inJournal
2003
Temporal infomax leads to almost deterministic dynamical systems
In: Neurocomputing, 52 (2003) 4, pp. 461-466inJournal
2003
Temporal Infomax on Markov chains with input leads to finite state automata
In: Neurocomputing, 52 (2003) 4, pp. 431-436inJournal
2002
Repository Open Access
An information-geometric approach to a theory of pragmatic structuring
In: The annals of probability, 30 (2002) 1, pp. 416-436Preprint
2002
Repository Open Access
Information-theoretic grounding of finite automata in neural systems
inJournal
2002
Repository Open Access
Locality of global stochastic interaction in directed acyclic networks
In: Neural computation, 14 (2002) 12, pp. 2959-2980Academic
2001
