Information-Theoretic Reasoning in Causal Inference

It is common knowledge that causal conclusions can not be drawn from statistical data, e.g. from associations/correlations, alone. However, in recent years a causal theory framework has been developed by Pearl, Spirtes, Glymour, Scheines and their collaborators that allows for causal conclusions due to imposed postulates on the data generation process. The main postulate connects stochastic dependences among random variables to the existence of 'causal mechanisms' underlying the data. Causal hypotheses in this theory can be formalized as directed acyclic graphs or generalizations thereof. The goal of causal inference then is to reconstruct the class of graphs from data by testing the constraints on the observed data that they would imply. However, there is no known efficient way to describe all constraints resulting from a causal hypothesis.

In this project we explore to what extent information-theoretic quantities can be helpful in this reconstruction process. The underlying idea is that quantities based on the relative entropy (e.g. mutual information) provide a coarse/macroscopic view on probability distributions that leads to testable constraints on the data and is still finer than just looking only at the qualitative property of stochastic independence. Quantifying stochastic dependences using information theory also turns out to be useful as a measure of strength of a causal connections (information flow).