Transfer entropy in continuous time, with applications to jump and neural spiking processes

Transfer entropy has been used to quantify the directed flow of information between source and target variables in many complex systems. Originally formulated in discrete time, we provide a framework for considering transfer entropy in continuous time systems. By appealing to a measure theoretic formulation we generalise transfer entropy, describing it in terms of Radon-Nikodym derivatives between measures of complete path realisations. The resulting formalism introduces and emphasises the idea that transfer entropy is an expectation of an individually fluctuating quantity along a path, in the same way we consider the expectation of physical quantities such as work and heat. We recognise that transfer entropy is a quantity accumulated over a finite time interval, whilst permitting an associated instantaneous transfer entropy rate. We use this approach to produce an explicit form for the transfer entropy for pure jump processes, and highlight the simplified form in the specific case of point processes (frequently used in neuroscience to model neural spike trains). We contrast our approach with previous attempts to formulate information flow between continuous time point processes within a discrete time framework, which incur issues that our continuous time approach naturally avoids. Finally, we present two synthetic spiking neuron model examples to exhibit the pertinent features of our formalism, namely that the information flow for point processes consists of discontinuous jump contributions (at spikes in the target) interrupting a continuously varying contribution (relating to waiting times between target spikes).