Search

MiS Preprint Repository

Delve into the future of research at MiS with our preprint repository. Our scientists are making groundbreaking discoveries and sharing their latest findings before they are published. Explore repository to stay up-to-date on the newest developments and breakthroughs.

MiS Preprint
52/2002

Information-theoretic grounding of finite automata in neural systems

Thomas Wennekers and Nihat Ay

Abstract

We introduce a measure "stochastic interaction" that captures spatial and temporal signal properties in recurrent systems. The measure quantifies the Kullback-Leibler divergence of a Markov chain from a product of split chains for the single units. Maximization of stochastic interaction, also called "Temporal Infomax", is shown to induce almost deterministic dynamical systems for unconstrained Markov chains. If part of the units are clamped to prescribed stochastic processes providing external input, Temporal Infomax leads to finite automata, either completely deterministic or at most weakly non-deterministic. This way, computational capabilities may arise in neural systems.

Received:
Jun 26, 2002
Published:
Jun 26, 2002
PACS:
84.35.+i, 87.19.La, 02.50.Ga
Keywords:
information theory, markov chains, neural networks

Related publications

Preprint
2002 Repository Open Access
Thomas Wennekers and Nihat Ay

Information-theoretic grounding of finite automata in neural systems