This project aims at identifying means to reduce the search space in learning systems as one way to improve the corresponding learning processes. To this end, we study the geometric properties of various connectionistic models known within the field of machine learning using information geometry and algebraic statistics. Our goal is to find distinguished architectures of learning systems based on their expressive power and learning performance.
This kind of model selection is motivated by experimental and theoretical work on restricted Boltzmann machines and deep belief networks, popular learning systems which evermore demand a profound mathematical investigation.
This project targets especially the development of design principles for our embodied AI project.
Keywords: information geometry, algebraic statistics, machine learning.
Selection Criteria for Neuromanifolds of Stochastic Dynamics
Within many formal models of neuronal systems, individual neurons are modelled as nodes which receive inputs from other
nodes in a network and generate an output that can be stochastic in general. This way the dynamics of the whole
network can be described as a stochastic transition in each time step, mathematically formalized in terms of a stochastic matrix.
Well-known models of this kind are Boltzmann machines, their generalizations, and policy matrices within reinforcement learning.
In order to study such learning systems it is helpful to consider not only one stochastic matrix but a parametrized
family of matrices, which forms a geometric object, referred to as neuromanifold within information geometry. Learning crucially depends on the shape of the neuromanifold.
This information geometric view, which has been proposed by Amari, suggests to select appropriate neuromanifolds and to define
corresponding learning processes as gradient flows on these manifolds.
We do not only focus on manifolds that are directly induced by a neuronal model, but study general sets that satisfy natural optimality conditions.
Two dimensional sets containing all deterministic policies
Deterministic policies or near to deterministic policies are optimal for a variety of reinforcement learning problems, they represent dynamics with maximal predictive information as considered in robotics and also dynamics of neural networks with maximal network information flow.
It is always possible to construct a two dimensional set that reaches all deterministic policies and on which natural gradient optimization works very efficiently.
Learning
These videos show experiments comparing ordinary (magenta) and natural (green) gradient learning our two dimensional models. Left: a system with 3 inputs and 2 outputs. Right: a system with 4 inputs and 2 outputs.
The contour lines correspond to the long term expected reward. The natural gradient method reaches the optimal solution in both experiments.
Related Group Publications:
N. Ay, G. Montufar, J. Rauh,
Selection criteria for neuromanifolds of stochastic dynamics.
Advances in cognitive neurodynamics III,
(2012). MIS-Preprint 15/2011
[pdf]
G. Montúfar, J. Rauh, N. Ay,
Expressive Power and Approximation Errors of Restricted Boltzmann Machines.
Advances in neural information processing systems 24 : 25th Annual Conference on Neural Information Processing Systems 2011, Granada, Spain December 12th - 15th,
(2011).
[pdf]
MIS Preprint 27/2011
[pdf]
G. Montúfar, N. Ay,
Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines.
Neural Computation (2011) 23:5,
pp: 1306-1319
[pdf] MIS-Preprint 23/2010
[pdf]
G. Montúfar,
Mixture Decomposition of Distributions using a Decomposition of the Sample Space.
Kybernetika (2010),
Arxiv 1008.0204
[pdf] MIS-Preprint 2010/39
[pdf]