The following text from the website of the Complex Systems Society addresses the question "What are Complex Systems?" and perfectly describes the main concept that underlies this project: "The most famous quote about Complex Systems comes from Aristole who said that 'The whole is more than the sum of its parts'. Complex systems are systems where the collective behavior of their parts entails emergence of properties that can hardly, if not at all, be inferred from properties of the parts."

We propose a geometric formalization of this concept. The complexity of a system is quantified as its deviation from the sum of its parts which is interpreted as a geometric projection. While our initial approach was based on information geometry only, the current study also applies the theory of hierarchical and, in particular, graphical models and causality theory based on Bayesian networks. This allows for an integrated analysis of the interplay of causal interactions, stochastic dependence, and complexity.

Relations to and among other approaches to complexity are studied. We are particularly interested in understanding how algorithmic notions of complexity correspond to probabilistic ones, similar to the well-known close connection between algorithmic complexity and Shannon entropy. In that context, various complexity measures for stochastic processes are related to corresponding complexities of typical process realizations, thereby identifying similarities of seemingly different concepts.

Related Group Publications:
Kanwal, M. S. ; Grochow, J. A. and N. Ay: Comparing information-theoretic measures of complexity in Boltzmann machines. Entropy, 19 (2017) 7, 310Bibtex [DOI] [ARXIV]

Perrone, P. and N. Ay: Hierarchical quantification of synergy in channels. Frontiers in robotics and AI, 2 (2016), 35Bibtex MIS-Preprint: 86/2015 [DOI] [ARXIV]

Perrone, P. and N. Ay: Iterative scaling algorithm for channels. Bibtex[ARXIV]

Ay, N. : Information geometry on complexity and stochastic interaction. Entropy, 17 (2015) 4, p. 2432-2458Bibtex MIS-Preprint: 95/2001 [DOI]

Bernigau, H. ; Kastoryano, M. J. and J. Eisert: Mutual information area laws for thermal free fermions. Journal of statistical mechanics, 2015 (2015), P02008Bibtex [DOI] [ARXIV]

Montúfar, G. and J. Rauh: Mode poset probability polytopes. Bibtex MIS-Preprint: 22/2015 [ARXIV]

Pfante, O. and N. Ay: Operator-theoretic identification of closed sub-systems of dynamical systems. An interdisciplinary journal of discontinuity, nonlinearity, and complexity, 4 (2015) 1, p. 91-109Bibtex MIS-Preprint: 4/2015 [DOI]

Weis, S. : The MaxEnt extension of a quantum Gibbs family, convex geometry and geodesics. Bayesian inference and maximum entropy methods in science and engineering : (MaxEnt 2014) : Clos Lucé, Amboise, France, 21-26 September 2014 / A. Mohammad-Djafari (ed.). AIP Publising, 2015. - P. 173-180 (AIP conference proceedings ; 1641) Bibtex [DOI] [ARXIV]

Bertschinger, N. and J. Rauh: The Blackwell relation defines no lattice. IEEE international symposium on information theory proceedings (ISIT) 2014 : June 29-July 4, 2014 in Honolulu, USA IEEE, 2014. - P. 2479-2483Bibtex [DOI] [ARXIV]

Bertschinger, N. ; Rauh, J. ; Olbrich, E. ; Jost, J. and N. Ay: Quantifying unique information. Entropy, 16 (2014) 4, p. 2161-2183Bibtex MIS-Preprint: 102/2013 [DOI] [ARXIV]

Rauh, J. ; Bertschinger, N. ; Olbrich, E. and J. Jost: Reconsidering unique information : towards a multivariate information decomposition. IEEE international symposium on information theory proceedings (ISIT) 2014 : June 29-July 4, 2014 in Honolulu, USA IEEE, 2014. - P. 2232-2236Bibtex [DOI] [ARXIV]

Bertschinger, N. ; Rauh, J. ; Olbrich, E. and J. Jost: Shared information : new insights and problems in decomposing information in complex systems. Proceedings of the European Conference on Complex Systems 2012 / T. Gilbert... (eds.). Springer, 2013. - P. 251-269 (Springer proceedings in complexity) Bibtex [DOI] [ARXIV]

Löhr, W. : Predictive models and generative complexity. Journal of systems science and complexity, 25 (2012) 1, p. 30-45Bibtex [DOI]

Löhr, W. ; Szkoła, A. and N. Ay: Process dimension of classical and non-commutative processes. Open systems and information dynamics, 19 (2012) 1, 1250007Bibtex MIS-Preprint: 52/2011 [DOI] [ARXIV]

Ay, N. ; Müller, M. and A. Szkoła: Effective complexity of stationary process realizations. Entropy, 13 (2011) 6, p. 1200-1211Bibtex MIS-Preprint: 2/2010 [DOI] [ARXIV]

Ay, N. ; Olbrich, E. ; Bertschinger, N. and J. Jost: A geometric approach to complexity. Chaos, 21 (2011) 3, 037103Bibtex MIS-Preprint: 53/2011 [DOI]

Campbell-Borges, Y. C. and J. Roberto C. Piqueira: Classical hierarchical correlation quantification on tripartite qubit mixed state families. Bibtex[ARXIV]

Ay, N. ; Müller, M. and A. Szkoła: Effective complexity and its relation to logical depth. IEEE transactions on information theory, 56 (2010) 9, p. 4593-4607Bibtex [DOI] [ARXIV]

Löhr, W. : Models of discrete-time stochastic processes and associated complexity measures. Dissertation, Universität Leipzig, 2010Bibtex[FREELINK]

Olbrich, E. ; Kahle, T. ; Bertschinger, N. ; Ay, N. and J. Jost: Quantifying structure in networks. The European physical journal / B, 77 (2010) 2, p. 239-247Bibtex MIS-Preprint: 81/2009 [DOI] [ARXIV]

Kahle, T. ; Olbrich, E. ; Jost, J. and N. Ay: Complexity measures from interaction structures. Physical review / E, 79 (2009) 2, pt. 2, 026201Bibtex MIS-Preprint: 44/2008 [DOI] [ARXIV]

Löhr, W. : Properties of the statistical complexity functional and partially deterministic HMMs. Entropy, 11 (2009) 3, p. 385-401Bibtex MIS-Preprint: 24/2009 [DOI]

Löhr, W. and N. Ay: On the generative nature of prediction. Advances in complex systems, 12 (2009) 2, p. 169-194Bibtex MIS-Preprint: 8/2008 [DOI]

Löhr, W. and N. Ay: Non-sufficient memories that are sufficient for prediction. Complex sciences : first international conference, Complex 2009, Shanghai, China, February 23 - 25, 2009, revised papers. Pt. 1 / J. Zhou (ed.). Springer, 2009. - P. 265-276 (Lecture notes of the Institute for Computer Science, Social Informatics and Telecommunications Engineering ; 4) Bibtex [DOI]

Olbrich, E. ; Bertschinger, N. ; Ay, N. and J. Jost: How should complexity scale with system size?. The European physical journal / B, 63 (2008) 3, p. 407-415Bibtex [DOI]

Wennekers, T. ; Ay, N. and P. Andras: High-resolution multiple-unit EEG in cat auditory cortex reveals large spatio-temporal stochastic interactions. Biosystems, 89 (2007) 1/3, p. 190-197Bibtex [DOI]

Ay, N. and A. Knauf: Maximizing multi-information. Kybernetika, 42 (2006) 5, p. 517-538Bibtex MIS-Preprint: 42/2003 [ARXIV]

Ay, N. ; Olbrich, E. ; Bertschinger, N. and J. Jost: A unifying framework for complexity measures of finite systems. ECCS'06 : proceedings of the European Conference on Complex Systems 2006 ; towards a science of complex systems / J. Jost... (eds.). European Complex Systems Society, 2006. - P. 80-80Bibtex[FREELINK]

Wennekers, T. and N. Ay: A temporal learning rule in recurrent systems supports high spatio-temporal stochastic interactions. Neurocomputing, 69 (2006) 10/12, p. 1199-1202Bibtex [DOI]

Wennekers, T. and N. Ay: Finite state automata resulting from temporal information maximization and a temporal learning rule. Neural computation, 17 (2005) 10, p. 2258-2290Bibtex [DOI]

Wennekers, T. and N. Ay: Stochastic interaction in associative nets. Neurocomputing, 65 (2005), p. 387-392Bibtex [DOI]

Erb, I. and N. Ay: Multi-information in the thermodynamic limit. Journal of statistical physics, 115 (2004) 3-4, p. 949-976Bibtex MIS-Preprint: 58/2003 [DOI]

Ay, N. and T. Wennekers: Dynamical properties of strongly interacting Markov chains. Neural networks, 16 (2003) 10, p. 1483-1497Bibtex MIS-Preprint: 107/2001 [DOI]

Ay, N. and T. Wennekers: Temporal infomax leads to almost deterministic dynamical systems. Neurocomputing, 52 (2003) 4, p. 461-466Bibtex [DOI]

Wennekers, T. and N. Ay: Temporal Infomax on Markov chains with input leads to finite state automata. Neurocomputing, 52 (2003) 4, p. 431-436Bibtex [DOI]

Wennekers, T. and N. Ay: Spatial and temporal stochastic interaction in neuronal assemblies. Theory in biosciences, 122 (2003) 1, p. 5-18Bibtex [DOI]

Wennekers, T. and N. Ay: Information-theoretic grounding of finite automata in neural systems. Bibtex MIS-Preprint: 52/2002

Ay, N. : Aspekte einer Theorie pragmatischer Informationsstrukturierung. Dissertation, Universität Leipzig, 2001Bibtex