Search

MiS Preprint Repository

We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV (www.arxiv.org) that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.

MiS Preprint
86/2015

Hierarchical Quantification of Synergy in Channels

Paolo Perrone and Nihat Ay

Abstract

The decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems.

Most approaches to the problem are based on information theory, and propose decompositions of mutual information between inputs and outputs in several ways, none of which is generally accepted yet.

We propose a new point of view on the topic. We model a multi-input channel as a Markov kernel. We can project the channel onto a series of exponential families which form a hierarchical structure. This is carried out with tools from information geometry, in a way analogous to the projections of probability distributions introduced by Amari. A Pythagorean relation leads naturally to a decomposition of the mutual information between inputs and outputs into terms which represent single node information; pairwise interactions; and in general n-node interactions.

The synergy measures introduced in this paper can be easily evaluated by an iterative scaling algorithm, which is a standard procedure in information geometry.

Received:
Dec 11, 2015
Published:
Dec 11, 2015
Keywords:
synergy, information decomposition, Divergences

Related publications

inJournal
2016 Repository Open Access
Paolo Perrone and Nihat Ay

Hierarchical quantification of synergy in channels

In: Frontiers in robotics and AI, 2 (2016), p. 35