Properties of the Statistical Complexity Functional and Partially Deterministic HMMs

Wolfgang Löhr

Contact the author: Please use for correspondence this email.
Submission date: 15. Jun. 2009 (revised version: August 2009)
Pages: 16
published in: Entropy, 11 (2009) 3, p. 385-401 
DOI number (of the published article): 10.3390/e110300385
Bibtex
Keywords and phrases: statistical complexity, prediction process, lower semi-continuity, ergodic decomposition, concavity, partially deterministic HMM
Download full preprint: PDF (251 kB)

Abstract:
Statistical complexity is a measure of complexity of discrete-time stationary stochastic processes, which has many applications. We investigate its more abstract properties as a non-linear functional on the space of processes and show its close relation to Knight's prediction process. We prove lower semi-continuity, concavity, and a formula for the ergodic decomposition of statistical complexity. On the way, we show that the discrete version of the prediction process has a continuous Markov transition. We also prove that, given the past output of a partially deterministic hidden Markov model (HMM), the uncertainty of the internal state is constant over time and knowledge of the internal state gives no additional information on the future output. Using this fact, we show that the causal state distribution is the unique stationary representation on prediction space that may have finite entropy.