MiS Preprint Repository

We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV ( that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.

MiS Preprint

Information-theoretic grounding of finite automata in neural systems

Thomas Wennekers and Nihat Ay


We introduce a measure "stochastic interaction" that captures spatial and temporal signal properties in recurrent systems. The measure quantifies the Kullback-Leibler divergence of a Markov chain from a product of split chains for the single units. Maximization of stochastic interaction, also called "Temporal Infomax", is shown to induce almost deterministic dynamical systems for unconstrained Markov chains. If part of the units are clamped to prescribed stochastic processes providing external input, Temporal Infomax leads to finite automata, either completely deterministic or at most weakly non-deterministic. This way, computational capabilities may arise in neural systems.

84.35.+i, 87.19.La, 02.50.Ga
information theory, markov chains, neural networks

Related publications

2002 Repository Open Access
Thomas Wennekers and Nihat Ay

Information-theoretic grounding of finite automata in neural systems