Quantifying unique information

Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay

Contact the author: Please use for correspondence this email.
Submission date: 13. Nov. 2013
Pages: 26
published in: Entropy, 16 (2014) 4, p. 2161-2183 
DOI number (of the published article): 10.3390/e16042161
Bibtex
MSC-Numbers: 94A15, 94A17
Keywords and phrases: Shannon information, mutual information, information decomposition, synergy
Download full preprint: PDF (402 kB)

Abstract:
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables (Y,Z) with a third random variable X. Our measures are motivated by an operational idea of unique information which suggests that shared information and unique information should depend only on the pair marginal distributions of (X,Y ) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.