The following text from the website of the Complex Systems Society addresses the question "What are Complex Systems?" and perfectly describes the main concept that underlies this project: "The most famous quote about Complex Systems comes from Aristole who said that 'The whole is more than the sum of its parts'. Complex systems are systems where the collective behavior of their parts entails emergence of properties that can hardly, if not at all, be inferred from properties of the parts."

We propose a geometric formalization of this concept. The complexity of a system is quantified as its deviation from the sum of its parts which is interpreted as a geometric projection. While our initial approach was based on information geometry only, the current study also applies the theory of hierarchical and, in particular, graphical models and causality theory based on Bayesian networks. This allows for an integrated analysis of the interplay of causal interactions, stochastic dependence, and complexity.

Relations to and among other approaches to complexity are studied. We are particularly interested in understanding how algorithmic notions of complexity correspond to probabilistic ones, similar to the well-known close connection between algorithmic complexity and Shannon entropy. In that context, various complexity measures for stochastic processes are related to corresponding complexities of typical process realizations, thereby identifying similarities of seemingly different concepts.

Related Group Publications:
Ay, N. ; Polani, D. and N. Virgo: Information decomposition based on cooperative game theory. Kybernetika, 56 (2020) 5, p. 979-1014 Bibtex MIS-Preprint: 81/2020 [DOI] [ARXIV]

Langer, C. and N. Ay: Complexity as causal information integration. Entropy, 22 (2020) 10, 1107 Bibtex MIS-Preprint: 85/2020 [DOI] [ARXIV]

Tirnakli, U. ; Tsallis, C. and N. Ay: Approaching a large deviation theory for complex systems. Bibtex [ARXIV]

Felice, D. and N. Ay: Divergence functions in information geometry. Geometric science of information : 4th international conference, GSI 2019, Toulouse, France, August 27-29, 2019, proceedings / F. Nielsen... (eds.). Springer, 2019. - P. 433-442 (Lecture notes in computer science ; 11712) Bibtex [DOI] [ARXIV]

Felice, D. and N. Ay: Canonical divergence for flat \(\alpha\)-connections : classical and quantum. Entropy, 21 (2019) 9, 831 Bibtex [DOI] [ARXIV]

Felice, D. ; Mancini, S. and N. Ay: Canonical divergence for measuring classical and quantum complexity. Entropy, 21 (2019) 4, 435 Bibtex [DOI] [ARXIV]

Banerjee, P. K. ; Rauh, J. and G. Montúfar: Computing the unique information. IEEE international symposium on information theory (ISIT) from June 17 to 22, 2018 at the Talisa Hotel in Vail, Colorado, USA IEEE, 2018. - P. 141-145 Bibtex MIS-Preprint: 73/2017 [DOI] [ARXIV] [CODELINK]

Felice, D. and N. Ay: Towards a canonical divergence within information geometry. Information geometry, Vol. not yet known, pp. not yet known Bibtex MIS-Preprint: 43/2018 [ARXIV]

Felice, D. and N. Ay: Dynamical systems induced by canonical divergence in dually flat manifolds. Information geometry, Vol. not yet known, pp. not yet known Bibtex MIS-Preprint: 103/2018 [ARXIV]

Felice, D. ; Cafaro, C. and S. Mancini: Information geometric methods for complexity. Chaos, 28 (2018) 3, 032101 Bibtex [DOI] [ARXIV]

Langer, C. and N. Ay: Comparison and connection between the joint and the conditional generalized iterative scaling algorithm. Proceedings of the 11th workshop on uncertainty processing WUPES '18, June 6-9, 2018 / V. Kratochvíl (ed.). MatfyzPress, 2018. - P. 105-116 Bibtex [FREELINK]

Rexiti, M. ; Felice, D. and S. Mancini: The volume of two-qubit states by information geometry. Entropy, 20 (2018) 2, 146 Bibtex [DOI] [ARXIV]

Kanwal, M. S. ; Grochow, J. A. and N. Ay: Comparing information-theoretic measures of complexity in Boltzmann machines. Entropy, 19 (2017) 7, 310 Bibtex [DOI] [ARXIV]

Perrone, P. and N. Ay: Hierarchical quantification of synergy in channels. Frontiers in robotics and AI, 2 (2016), 35 Bibtex MIS-Preprint: 86/2015 [DOI] [ARXIV]

Perrone, P. and N. Ay: Iterative scaling algorithm for channels. Bibtex [ARXIV]

Pfante, O. ; Bertschinger, N. ; Olbrich, E. ; Ay, N. and J. Jost: Wie findet man eine geeignete Beschreibungsebene für ein komplexes System?. Jahrbuch der Max-Planck-Gesellschaft, 2016 (2016), Forschungsbericht - Max-Planck-Institut für Mathematik in den Naturwissenschaften Bibtex [FREELINK]

Ay, N.: Information geometry on complexity and stochastic interaction. Entropy, 17 (2015) 4, p. 2432-2458 Bibtex MIS-Preprint: 95/2001 [DOI]

Bernigau, H. ; Kastoryano, M. J. and J. Eisert: Mutual information area laws for thermal free fermions. Journal of statistical mechanics, 2015 (2015) 2, P02008 Bibtex [DOI] [ARXIV]

Montúfar, G. and J. Rauh: Mode poset probability polytopes. Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / V. Kratochvíl (ed.). Oeconomica, 2015. - P. 147-154 Bibtex MIS-Preprint: 22/2015 [ARXIV] [FREELINK]

Perrone, P. and N. Ay: Decomposition of Markov kernels. Proceedings of the 10th workshop on uncertainty processing WUPES '15, Moninec, Czech Republic, September 16-19, 2015 / V. Kratochvíl (ed.). Oeconomica, 2015. - P. 167-178 Bibtex [FREELINK]

Pfante, O. and N. Ay: Operator-theoretic identification of closed sub-systems of dynamical systems. An interdisciplinary journal of discontinuity, nonlinearity, and complexity, 4 (2015) 1, p. 91-109 Bibtex MIS-Preprint: 4/2015 [DOI]

Weis, S.: The MaxEnt extension of a quantum Gibbs family, convex geometry and geodesics. Bayesian inference and maximum entropy methods in science and engineering : (MaxEnt 2014) : Clos Lucé, Amboise, France, September 21-26 2014 / A. Mohammad-Djafari (ed.). AIP Publising, 2015. - P. 173-180 (AIP conference proceedings ; 1641) Bibtex [DOI] [ARXIV]

Bertschinger, N. and J. Rauh: The Blackwell relation defines no lattice. IEEE international symposium on information theory proceedings (ISIT) 2014 : June 29-July 4, 2014 in Honolulu, USA IEEE, 2014. - P. 2479-2483 Bibtex [DOI] [ARXIV]

Bertschinger, N. ; Rauh, J. ; Olbrich, E. ; Jost, J. and N. Ay: Quantifying unique information. Entropy, 16 (2014) 4, p. 2161-2183 Bibtex MIS-Preprint: 102/2013 [DOI] [ARXIV] [CODELINK]

Pfante, O. ; Bertschinger, N. ; Olbrich, E. ; Ay, N. and J. Jost: Comparison between different methods of level identification. Advances in complex systems, 17 (2014) 2, 1450007 Bibtex [DOI]

Pfante, O. ; Olbrich, E. ; Bertschinger, N. ; Ay, N. and J. Jost: Closure measures for coarse-graining of the tent map. Chaos, 24 (2014), 013136 Bibtex MIS-Preprint: 108/2013 [DOI]

Rauh, J. ; Bertschinger, N. ; Olbrich, E. and J. Jost: Reconsidering unique information : towards a multivariate information decomposition. IEEE international symposium on information theory proceedings (ISIT) 2014 : June 29-July 4, 2014 in Honolulu, USA IEEE, 2014. - P. 2232-2236 Bibtex [DOI] [ARXIV]

Bertschinger, N. ; Rauh, J. ; Olbrich, E. and J. Jost: Shared information : new insights and problems in decomposing information in complex systems. Proceedings of the European Conference on Complex Systems 2012 / T. Gilbert... (eds.). Springer, 2013. - P. 251-269 (Springer proceedings in complexity) Bibtex [DOI] [ARXIV]

Löhr, W.: Predictive models and generative complexity. Journal of systems science and complexity, 25 (2012) 1, p. 30-45 Bibtex [DOI]

Löhr, W. ; Szkoła, A. and N. Ay: Process dimension of classical and non-commutative processes. Open systems and information dynamics, 19 (2012) 1, 1250007 Bibtex MIS-Preprint: 52/2011 [DOI] [ARXIV]

Ay, N. ; Müller, M. and A. Szkoła: Effective complexity of stationary process realizations. Entropy, 13 (2011) 6, p. 1200-1211 Bibtex MIS-Preprint: 2/2010 [DOI] [ARXIV]

Ay, N. ; Olbrich, E. ; Bertschinger, N. and J. Jost: A geometric approach to complexity. Chaos, 21 (2011) 3, 037103 Bibtex MIS-Preprint: 53/2011 [DOI]

Campbell-Borges, Y. C. and J. Roberto C. Piqueira: Classical hierarchical correlation quantification on tripartite qubit mixed state families. Bibtex [ARXIV]

Ay, N. ; Müller, M. and A. Szkoła: Effective complexity and its relation to logical depth. IEEE transactions on information theory, 56 (2010) 9, p. 4593-4607 Bibtex [DOI] [ARXIV]

Löhr, W.: Models of discrete-time stochastic processes and associated complexity measures. Dissertation, Universität Leipzig, 2010 Bibtex [FREELINK]

Olbrich, E. ; Kahle, T. ; Bertschinger, N. ; Ay, N. and J. Jost: Quantifying structure in networks. The European physical journal / B, 77 (2010) 2, p. 239-247 Bibtex MIS-Preprint: 81/2009 [DOI] [ARXIV]

Kahle, T. ; Olbrich, E. ; Jost, J. and N. Ay: Complexity measures from interaction structures. Physical review / E, 79 (2009) 2, pt. 2, 026201 Bibtex MIS-Preprint: 44/2008 [DOI] [ARXIV]

Löhr, W.: Properties of the statistical complexity functional and partially deterministic HMMs. Entropy, 11 (2009) 3, p. 385-401 Bibtex MIS-Preprint: 24/2009 [DOI]

Löhr, W. and N. Ay: On the generative nature of prediction. Advances in complex systems, 12 (2009) 2, p. 169-194 Bibtex MIS-Preprint: 8/2008 [DOI]

Löhr, W. and N. Ay: Non-sufficient memories that are sufficient for prediction. Complex sciences : first international conference, Complex 2009, Shanghai, China, February 23 - 25, 2009, revised papers. Pt. 1 / J. Zhou (ed.). Springer, 2009. - P. 265-276 (Lecture notes of the Institute for Computer Science, Social Informatics and Telecommunications Engineering ; 4) Bibtex [DOI]

Olbrich, E. ; Bertschinger, N. ; Ay, N. and J. Jost: How should complexity scale with system size?. The European physical journal / B, 63 (2008) 3, p. 407-415 Bibtex [DOI]

Wennekers, T. ; Ay, N. and P. Andras: High-resolution multiple-unit EEG in cat auditory cortex reveals large spatio-temporal stochastic interactions. Biosystems, 89 (2007) 1/3, p. 190-197 Bibtex [DOI]

Ay, N. and A. Knauf: Maximizing multi-information. Kybernetika, 42 (2006) 5, p. 517-538 Bibtex MIS-Preprint: 42/2003 [ARXIV]

Ay, N. ; Olbrich, E. ; Bertschinger, N. and J. Jost: A unifying framework for complexity measures of finite systems. ECCS'06 : proceedings of the European Conference on Complex Systems 2006 ; towards a science of complex systems / J. Jost... (eds.). European Complex Systems Society, 2006. - P. 80-80 Bibtex [FREELINK]

Wennekers, T. and N. Ay: A temporal learning rule in recurrent systems supports high spatio-temporal stochastic interactions. Neurocomputing, 69 (2006) 10/12, p. 1199-1202 Bibtex [DOI]

Wennekers, T. and N. Ay: Finite state automata resulting from temporal information maximization and a temporal learning rule. Neural computation, 17 (2005) 10, p. 2258-2290 Bibtex [DOI]

Wennekers, T. and N. Ay: Stochastic interaction in associative nets. Neurocomputing, 65 (2005), p. 387-392 Bibtex [DOI]

Erb, I. and N. Ay: Multi-information in the thermodynamic limit. Journal of statistical physics, 115 (2004) 3-4, p. 949-976 Bibtex MIS-Preprint: 58/2003 [DOI]

Ay, N. and T. Wennekers: Dynamical properties of strongly interacting Markov chains. Neural networks, 16 (2003) 10, p. 1483-1497 Bibtex MIS-Preprint: 107/2001 [DOI]

Ay, N. and T. Wennekers: Temporal infomax leads to almost deterministic dynamical systems. Neurocomputing, 52 (2003) 4, p. 461-466 Bibtex [DOI]

Wennekers, T. and N. Ay: Temporal Infomax on Markov chains with input leads to finite state automata. Neurocomputing, 52 (2003) 4, p. 431-436 Bibtex [DOI]

Wennekers, T. and N. Ay: Spatial and temporal stochastic interaction in neuronal assemblies. Theory in biosciences, 122 (2003) 1, p. 5-18 Bibtex [DOI]

Wennekers, T. and N. Ay: Information-theoretic grounding of finite automata in neural systems. Bibtex MIS-Preprint: 52/2002

Ay, N.: Aspekte einer Theorie pragmatischer Informationsstrukturierung. Dissertation, Universität Leipzig, 2001 Bibtex