Shun-ichi Amari
RIKEN, Japan
Information geometry and its applications to learning

Information geometry studies the invariant geometrical structure of a manifold of probability distributions. It consists of Riemannian metric due to Fisher information and a pair of dually coupled affine connections. Since learning takes place under stochastic environments, it provides a useful new tool to various aspects of learning. The present talk begins with an intuitive introduction to information geometry, where prior knowledge on differential geometry is not required. After introducing nice properties such as the generalized Pythagorean theorem, various applications to machine learning and statistical inference are demonstrated.

Shun-ichi Amari
RIKEN, Japan
Basic principles of supervised and unsupervised learning: toward understanding deep learning

Both supervised and unsupervised techniques are used in Deep Learning. We show a simple model of self-organization (Hebbian learning) to understand how new representations of signals are organized. We then show self-organization of Restricted Boltzmann machines and auto-encoder (recurrently connected neural networks). They reveal the way of information representations in a hierarchical system. The multilayer perceptron is used for supervised learning. However, its space of parameters includes lots of singularities, which cause difficulty in learning. We analyze the dynamics of supervised learning near singularities, and demonstrate that the natural gradient descent learning method improves the efficiency of learning largely.

Tamim Asfour
KIT Karlsruhe, Germany
Structural bootstrapping for 24/7 humanoids

Humanoid robotics has made significant progress and will continue to play a central role in robotics research and many applications of the 21st century. Ambitious goals have been set for future humanoid robotics. They are expected to serve as companions and assistants for humans in daily life and as ultimate helpers in man-made and natural disasters. Although current humanoids are technologically advanced, they are still limited in their actuation, sensing, prediction, interaction, and learning capabilities. Versatile humanoid robot systems integrating perception, action, prediction, planning, and lifelong learning to carry out tasks in 24/7 manner in the real world are still missing.
The first part of the talk will present recent progress towards building 24/7 humanoid robots able to predict and act in the real world, to interact and collaborate with humans in human-centered environments, to autonomously acquire knowledge about objects and their own bodies through active visuo-haptic exploration, and to learn and imitate human actions. The capabilities will be demonstrated on the humanoid robots ARMAR-IIIa and ARMAR-IIIb. The second part of the talk will introduce and discuss the concept of Structural Bootstrapping, an idea taken from child language acquisition research, for building generative models, leveraging existing experience to predict unexplored action effects, and to focus the hypothesis space for learning novel concepts. Structural bootstrapping makes use of two different kinds of information about concepts, words or actions to speed up learning. Specifically, structural bootstrapping leverages information about how concepts, words, or actions are used (their syntax) against the concept, word, or action’s meaning (its semantics) to learn new concepts, words, and actions.

Michael Beetz
Bremen University, Germany
Autonomous Learning for Human-scale Everyday Manipulation Tasks

Despite the fact that autonomous robotic agents performing human-scale manipulation tasks need to learn vast amounts of knowledge and many different skills the application domain receives surprisingly little attention in the area autonomous learning. On the other hand, the knowledge intensive character and the complexity of tasks as well as the desired level of performance require autonomous learning to include mechanisms that go well beyond the current state-of-the-art.

Bayesian cognitive robotics is a novel paradigm for the knowledge-enabled control of autonomous robots. The paradigm presumes that one of the most powerful ideas to equip robots with comprehensive reasoning capabilities is the lifelong autonomous learning of joint probability distributions over robot control programs, the behavior they generate and the situation-dependent effects they bring about. Having learned such probability distributions from experience, a robot can make predictions, diagnoses and perform other valuable inference tasks in order to improve its problem-solving performance.

In this talk, I will describe and discuss

1. techniques for embodying methods of Bayesian cognitive robotics into modern autonomous robot control systems performing human-scale manipulation tasks in real-world settings
2. learning techniques and mechanisms for scaling learning for more realistic domain sizes and producing knowledge that is applicable in perception-guided manipulation and
3. methods for applying the learned knowledge to substantially improve the problem-solving capabilities and performance of robotic agents.

Matthias Bethge
University of Tübingen, MPI for Biological Cybernetics, Bernstein Center for Computational Neuroscience, Germany
Natural image statistics & neural representation learning

An important motivation for studying the statistics of natural images is the search for image representations which facilitate visual inference tasks. Representations optimized directly for a given task are at risk of overfitting, that is, the representations might work well for that particular task but might not generalize well to others. However, the striking ability of our visual system to perform well in a variety of different situations and to recognize objects even when they have been seen only once suggests that it exploits general structural regularities of natural images. In this lecture, I will give an overview on natural image statistics and how different types of representations have been derived by modeling different statistical properties of natural images.

Christos Dimitrakakis
Chalmers University of Technology, Sweden
Planning under uncertainty: Markov decision processes

This tutorial will give an introduction to decision theory and reinforcement learning. Starting from an introduction to preferences and utility, we will then cover sequential decision problems. These can be formalised as Markov decision processes. Within this framework lie many important problems such as adaptive experiment design and reinforcement learning. We will also discuss some foundational algorithms and models for reinforcement learning.

Keyan Ghazi-Zahedi
Max Planck Institute for Mathematics in the Sciences, Germany
Intrinsically motivated exploration of behavioural modes

In the context of machine learning, acting to learn is most often associated with the exploration of the world model, which means that actions are performed that minimise the error of the internal representation of the external world dynamics. This essential form of acting to learn will be discussed within the first lectures of this series. This lecture will focus on a different aspect of acting to learn, which is important in the context of embodied intelligence, namely the autonomous learning of behavioral modes. The different walking gaits of a horse are a good illustration for what we describe as a behavioural mode. It is a behavior that is compliant with the body and the environment of the system, or in other words, which exploits the physical properties of the systems body and environment such that it is optimal with respect to some criteria (e.g. energy consumption). The main focus of this lecture is the question how a system, that is initialised with minimal knowledge, can autonomously gather useful information about itself and its habitat. Examples are human infants, which have to learn their behavioral modes, e.g. grasping and walking, without any direct support by an external teacher. Although parents may prevent their children from falling, they cannot actually teach them how to walk. This form of learning must be a self-organised process. In this last lecture of the acting to learn series, we will discuss the maximisation of the predictive information as an information-driven, self-organising learning principle which leads to an autonomous learning of behavioral modes.

The lecture consists of four parts. After motivating the question in more detail, the mathematical framework will be introduced as far as it is required to understand this lecture without any a priori knowledge about information theory or the previous lectures in this series. In the second part, we will discuss the causal model of the sensorimotor loop and its utility in the context of embodied artificial intelligence. As an example, the quantification of causal information flows between actions and sensations will be derived along this model. The third part of this lecture will introduce the predictive information and its application to the sensorimotor loop. We will apply Amariś natural gradient method to derive a policy gradient for the maximisation of the predictive information and apply it to embodied systems in the last part of this lecture. The discussion of the results will also lead to insights about morphological computation.

Thomas Martinetz
University of Lübeck, Germany
Sparse coding and efficient sensing

For being able to act autonomously in its world, an agent has to learn appropriate internal representations. In the brain we find simple and complex cells and the concept of sparse coding. Sparse coding seems to mirror the structure of natural scenes and signals, and the combination of simple and complex cells can provide invariant representations. We will give an introduction to sparse coding and show how the state of a sparsely encodeable world can be sensed very efficiently, e.g. by compressive sensing or adaptive hierarchical sensing. We will show how abstract and generalized simple and complex cells lead to representations which are invariant to a large class of transformations, which is the basis for the success of deep convolutional networks. We will see that deep convolutional networks with their simple and complex cells favor sparse codes.

Helge Ritter
Bielefeld University, Germany
Issues, algorithms, and challenges
Learning appears as one of the most fascinating aspects of cognition and as a hallmark of intelligence -- be it natural or artificial. However, learning as seen in cognitive agents that must act and survive in natural environments is often far from the crisp and idealized notions of learning that have become elaborated in machine learning. Here, the major dichotomy is to identify a mapping or a probability density, with a plethora of methods to represent, construct and evaluate these "objects" leading to a meanwhile richly differentiated spectrum of learning algorithms and their characterization.

Robotics exerts an increasing impact on this science by introducing a number of challenges that tended to by sidestepped in earlier machine learning work that circled mainly around the idea of "mappings" or probability densities to be estimated from large numbers of passively observed examples. First of all, robots with their closed sensory-motor loops require to consider learning when the data are not only observed but also changed or even created during acting, which necessitates a perspective shift from mappings to controllers or dynamical systems. Moreover, even our simplest daily actions connect different levels of learning that seem to coexist in real-world "cognitive" learners with their embarrassingly strong generalization abilities that enable them to learn from much fewer examples than most current machine learning algorithms would require. Finally, the grounding of robots in the real world that results from their sensors, actors and their body shape, is far from trivial and has a strong impact on what can be learnt and what needs to be learnt.

These and more factors make the interface between robotics and the fields of machine and of cognitive learning a rich area with exciting developments, scientific challenges and opportunities for research. The tutorial attempts a (highly selective) "tour" through some of the major issues with a discussion of the roles and lessons from robotics as a "connecting science" between the strands of machine learning and cognitive learning research. Along this way, we show and discuss examples from pertinent current work, including a status report and outlook on the recently launched project FAMULA that connects a number of CITEC research groups for developing a robot that employs manual action and language to familiarize itself with novel objects.

Friedrich Sommer
Redwood Center for Theoretical Neuroscience, UC Berkeley, USA
Information-theory based policies for learning in (embodied) closed sensori-motor loops

Over the last two decades great progress has been made in understanding how sensory representations are learned in the brain driven by the principle of efficient coding. In contrast, we are still lacking theories of learning in closed sensor-motor loops is still lacking. My lecture will first review foundational work that defined information gain and proposed it for guiding optimal experimental design and for driving learning in action-perception loops. Second I will present more recent work using information gain for describing exploratory learning of agents in unknown environments combining optimizing information gain within a multi-step time horizon. Finally I will discuss the extension of this work to the exploration of unbounded state spaces.

Marc Toussaint
Stuttgart University, Germany
Bandits, global optimization, active learning, and Bayesian reinforcement learning -- understanding the common ground

In active learning and related scenarios the learning agent actively selects which data point should be labeled next. That is, instead of learning passively from a previously selected data set, the learning agent is in control of the data collection itself, with the aim of collecting the data which is most informative to the model. This can greatly reduce the number of necessary data points. In this tutorial I will give a basic introduction to this area, including:

• What is the relation between bandits, optimization, active learning and Bayesian reinforcement learning?
• What would ”optimal active learning” be?
• What are typical and efficient heuristics (including regret bounds)?
• What are modern Monte-Carlo tree search approaches?