Abstract for the talk at 23.09.2014 (15:15 h)


Johannes Rauh (MPI MIS, Leipzig)
Synergy and redundancy - decomposing the multivariate mutual information

Consider three random variables X,Y,Z taking values in finite sets. According to Shannon, the information that a measurement of Y gives about the outcome of X can be quantified by the mutual information I(X : Y ). In the presence of the third variable Z, one can ask whether Z contains the same information (redundant information), whether Z contains additional information that Y did not have (unique information), or whether there is synergistic information that neither Y and Z alone contain, but that they can only access together (for example, if Y and Z are binary i.i.d. variables and if X = Y XORZ, then neither Y nor Z alone know anything about X, but together they can compute X). Mathematically, one would like to express this distinction as a decomposition of the total mutual information I(X : {Y,Z}) into a sum of different non-negative terms with a well-defined interpretation. At present, a consistent decomposition is not known. However, there are different proposed functions that try to measure one of these aspects (redundant, unique or synergistic information). For example, neuroscientists have tried for a long time to measure the synergy of the neurons in a network, but so far, no convincing measure has been found.

In my talk I present recent results of joint work with Nils Bertschinger, Eckehard Olbich, Jürgen Jost and Nihat Ay about a new proposed information decomposition. Our measures are motivated by an operational idea of unique information which suggests that shared information and unique information should depend only on the pair marginal distributions of (X,Y ) and (X,Z). This invariance property has not been studied before, but it is satisfied by other proposed measures of shared information. Although we could prove many nice properties of our functions, so far we do not know how to extend these results to the case of more than three random variables.


01.03.2017, 13:57