Institute
General information about the institute, such as our mission statement, organizational structure, staff directory, history, directions, etc.

See more
- About Us
- Visit Us
- Organization
- Directory
- Services
- Fresh Faces
Research
Scientific profile with all research groups, topics, collaborations, as well as columns on research at the institute.

See more
News
News and press releases about the institute, as well as a media archive.

See more
- News Overview
- Press Releases
Events
Overview of all events around the institute, such as talks, seminars, lectures, workshops, conferences and public events.

See more
Publications
Overview of all scientific publications of the institute, as well as our preprint and software repositories.

See more
Career
Information on open positions at the institute, benefits of working with us, graduate school, and postdoctoral supervision.

See more

Events
Autonomous Learning

conference

01/09/2014 04/09/2014

Autonomous Learning

MPI für Mathematik in den Naturwissenschaften Leipzig

E1 05 (Leibniz-Saal)

Autonomous Learning research aims at understanding how autonomous systems can efficiently learn from the interaction with the environment, especially by having an integrated approach to decision making and learning, allowing systems to autonomously decide on actions, representations, hyperparameters and model structures for the purpose of efficient learning.

In this summer school international and national experts will introduce to the core concepts and related theory for autonomous learning in real-world environments. We hope to foster the enthusiasm of young researchers for this exciting research area, giving them the opportunity to meet leading experts in the field and similarly interested students. Our school offers an opportunity to look into fundamental and advanced aspects of autonomous learning. The tutorials are structured around three themes:

learning representations,
acting to learn (exploration), and
learning to act in real-world environments.

The themes include but are not restricted to the following subjects:

Theme 1: Learning representations

compressed sensing/sparse coding
deep learning
hierarchical representations in perception
learning abstractions and symbols

Theme 2: Acting to learn (exploration)

foundations of optimal exploration and information seeking
approaches to intrinsic motivation

Theme 3: Learning to act in real-world environments

relational reinforcement learning
Bayesian cognitive robotics

Financial support

The summer school is financially supported by the <link http: autonomous-learning.org external>DFG Priority Program Autonomous Learning.

Application

Registration is possible with the application form on this website until May 31. We will ask you for a short CV with a description of your research interest and a reference contact (we will email your recommender to provide a brief letter). Applicants will be informed of the outcome of their application until June 17.

Accommodation and costs

This course is free of charge, but participants have to cover their own travel, room and board. We have blocked an affordable room allocation at A&O Hostel in Leipzig (Hauptbahnhof, Brandenburger Straße 2, 04103 Leipzig). More information by email after your application was accepted.

Speakers

Shun-ichi Amari

Tamim Asfour

Michael Beetz

Matthias Bethge

Christos Dimitrakakis

Keyan Ghazi-Zahedi

Thomas Martinetz

Helge Ritter

Satinder Singh

Friedrich Sommer

Marc Toussaint

Sep 1, 2014
10:30	Welcome address
11:00	Theme 1: Learning representations : Shun-ichi Amari (RIKEN, Japan) Information geometry and its applications to learning Information geometry studies the invariant geometrical structure of a manifold of probability distributions. It consists of Riemannian metric due to Fisher information and a pair of dually coupled affine connections. Since learning takes place under stochastic environments, it provides a useful new tool to various aspects of learning. The present talk begins with an intuitive introduction to information geometry, where prior knowledge on differential geometry is not required. After introducing nice properties such as the generalized Pythagorean theorem, various applications to machine learning and statistical inference are demonstrated. Video Slides
12:30	Lunch break
14:30	Theme 1: Learning representations: Shun-ichi Amari (RIKEN, Japan) Basic principles of supervised and unsupervised learning: toward understanding deep learning Both supervised and unsupervised techniques are used in Deep Learning. We show a simple model of self-organization (Hebbian learning) to understand how new representations of signals are organized. We then show self-organization of Restricted Boltzmann machines and auto-encoder (recurrently connected neural networks). They reveal the way of information representations in a hierarchical system. The multilayer perceptron is used for supervised learning. However, its space of parameters includes lots of singularities, which cause difficulty in learning. We analyze the dynamics of supervised learning near singularities, and demonstrate that the natural gradient descent learning method improves the efficiency of learning largely. Video Slides
17:30	Get-together BBQ at Glashaus (on individual payment)
Sep 2, 2014
09:00	Theme 3: Learning to act in real-world environments: Christos Dimitrakakis (Chalmers University of Technology, Gothenburg, Sweden) Planning under uncertainty: Markov decision processes This tutorial will give an introduction to decision theory and reinforcement learning. Starting from an introduction to preferences and utility, we will then cover sequential decision problems. These can be formalised as Markov decision processes. Within this framework lie many important problems such as adaptive experiment design and reinforcement learning. We will also discuss some foundational algorithms and models for reinforcement learning. Video Slides
10:30	Coffee break
11:00	Theme 2: Acting to learn (exploration) : Keyan Ghazi-Zahedi (Max Planck Institute for Mathematics in the Sciences, Germany) Intrinsically motivated exploration of behavioural modes In the context of machine learning, acting to learn is most often associated with the exploration of the world model, which means that actions are performed that minimise the error of the internal representation of the external world dynamics. This essential form of acting to learn will be discussed within the first lectures of this series. This lecture will focus on a different aspect of acting to learn, which is important in the context of embodied intelligence, namely the autonomous learning of behavioral modes. The different walking gaits of a horse are a good illustration for what we describe as a behavioural mode. It is a behavior that is compliant with the body and the environment of the system, or in other words, which exploits the physical properties of the system’s body and environment such that it is optimal with respect to some criteria (e.g. energy consumption). The main focus of this lecture is the question how a system, that is initialised with minimal knowledge, can autonomously gather useful information about itself and its habitat. Examples are human infants, which have to learn their behavioral modes, e.g. grasping and walking, without any direct support by an external teacher. Although parents may prevent their children from falling, they cannot actually teach them how to walk. This form of learning must be a self-organised process. In this last lecture of the acting to learn series, we will discuss the maximisation of the predictive information as an information-driven, self-organising learning principle which leads to an autonomous learning of behavioral modes. The lecture consists of four parts. After motivating the question in more detail, the mathematical framework will be introduced as far as it is required to understand this lecture without any a priori knowledge about information theory or the previous lectures in this series. In the second part, we will discuss the causal model of the sensorimotor loop and its utility in the context of embodied artificial intelligence. As an example, the quantification of causal information flows between actions and sensations will be derived along this model. The third part of this lecture will introduce the predictive information and its application to the sensorimotor loop. We will apply Amariś natural gradient method to derive a policy gradient for the maximisation of the predictive information and apply it to embodied systems in the last part of this lecture. The discussion of the results will also lead to insights about morphological computation. Video Slides
12:30	Lunch break
14:30	Theme 1: Learning representations: Thomas Martinetz (University of Lübeck, Germany) Sparse coding and efficient sensing For being able to act autonomously in its world, an agent has to learn appropriate internal representations. In the brain we find simple and complex cells and the concept of sparse coding. Sparse coding seems to mirror the structure of natural scenes and signals, and the combination of simple and complex cells can provide invariant representations. We will give an introduction to sparse coding and show how the state of a sparsely encodeable world can be sensed very efficiently, e.g. by compressive sensing or adaptive hierarchical sensing. We will show how abstract and generalized simple and complex cells lead to representations which are invariant to a large class of transformations, which is the basis for the success of deep convolutional networks. We will see that deep convolutional networks with their simple and complex cells favor sparse codes. Video Slides
16:00	Coffee break
16:30	Theme 2: Acting to learn (exploration) : Friedrich Sommer (Redwood Center for Theoretical Neuroscience, UC Berkeley, USA) Information-theory based policies for learning in (embodied) closed sensori-motor loops Over the last two decades great progress has been made in understanding how sensory representations are learned in the brain driven by the principle of efficient coding. In contrast, we are still lacking theories of learning in closed sensor-motor loops is still lacking. My lecture will first review foundational work that defined information gain and proposed it for guiding optimal experimental design and for driving learning in action-perception loops. Second I will present more recent work using information gain for describing exploratory learning of agents in unknown environments combining optimizing information gain within a multi-step time horizon. Finally I will discuss the extension of this work to the exploration of unbounded state spaces. Video
Sep 3, 2014
09:00	Theme 3: Learning to act in real-world environments : Tamim Asfour (KIT Karlsruhe, Germany) Structural bootstrapping for 24/7 humanoids Humanoid robotics has made significant progress and will continue to play a central role in robotics research and many applications of the 21st century. Ambitious goals have been set for future humanoid robotics. They are expected to serve as companions and assistants for humans in daily life and as ultimate helpers in man-made and natural disasters. Although current humanoids are technologically advanced, they are still limited in their actuation, sensing, prediction, interaction, and learning capabilities. Versatile humanoid robot systems integrating perception, action, prediction, planning, and lifelong learning to carry out tasks in 24/7 manner in the real world are still missing. The first part of the talk will present recent progress towards building 24/7 humanoid robots able to predict and act in the real world, to interact and collaborate with humans in human-centered environments, to autonomously acquire knowledge about objects and their own bodies through active visuo-haptic exploration, and to learn and imitate human actions. The capabilities will be demonstrated on the humanoid robots ARMAR-IIIa and ARMAR-IIIb. The second part of the talk will introduce and discuss the concept of Structural Bootstrapping, an idea taken from child language acquisition research, for building generative models, leveraging existing experience to predict unexplored action effects, and to focus the hypothesis space for learning novel concepts. Structural bootstrapping makes use of two different kinds of information about concepts, words or actions to speed up learning. Specifically, structural bootstrapping leverages information about how concepts, words, or actions are used (their syntax) against the concept, word, or action’s meaning (its semantics) to learn new concepts, words, and actions. Video Slides
10:30	Coffee break
11:00	Theme 2: Acting to learn (exploration) : Marc Toussaint (Stuttgart University, Germany) Bandits, global optimization, active learning, and Bayesian reinforcement learning -- understanding the common ground In active learning and related scenarios the learning agent actively selects which data point should be labeled next. That is, instead of learning passively from a previously selected data set, the learning agent is in control of the data collection itself, with the aim of collecting the data which is most informative to the model. This can greatly reduce the number of necessary data points. In this tutorial I will give a basic introduction to this area, including:What is the relation between bandits, optimization, active learning and Bayesian reinforcement learning?What would "optimal active learning'' be?What are typical and efficient heuristics (including regret bounds)?What are modern Monte-Carlo tree search approaches? Video Slides
12:30	Lunch break
14:30	Poster session
18:00	Social event
Sep 4, 2014
09:00	Theme 3: Learning to act in real-world environments: Michael Beetz (Bremen University, Germany) Autonomous Learning for Human-scale Everyday Manipulation Tasks Despite the fact that autonomous robotic agents performing human-scale manipulation tasks need to learn vast amounts of knowledge and many different skills the application domain receives surprisingly little attention in the area autonomous learning. On the other hand, the knowledge intensive character and the complexity of tasks as well as the desired level of performance require autonomous learning to include mechanisms that go well beyond the current state-of-the-art. Bayesian cognitive robotics is a novel paradigm for the knowledge-enabled control of autonomous robots. The paradigm presumes that one of the most powerful ideas to equip robots with comprehensive reasoning capabilities is the lifelong autonomous learning of joint probability distributions over robot control programs, the behavior they generate and the situation-dependent effects they bring about. Having learned such probability distributions from experience, a robot can make predictions, diagnoses and perform other valuable inference tasks in order to improve its problem-solving performance. In this talk, I will describe and discuss techniques for embodying methods of Bayesian cognitive robotics into modern autonomous robot control systems performing human-scale manipulation tasks in real-world settings learning techniques and mechanisms for scaling learning for more realistic domain sizes and producing knowledge that is applicable in perception-guided manipulation and methods for applying the learned knowledge to substantially improve the problem-solving capabilities and performance of robotic agents. Video Slides
10:30	Coffee break
11:00	Theme 3: Learning to act in real-world environments : Helge Ritter (Bielefeld University, Bielefeld, Germany) Issues, algorithms, and challenges Learning appears as one of the most fascinating aspects of cognition and as a hallmark of intelligence -- be it natural or artificial. However, learning as seen in cognitive agents that must act and survive in natural environments is often far from the crisp and idealized notions of learning that have become elaborated in machine learning. Here, the major dichotomy is to identify a mapping or a probability density, with a plethora of methods to represent, construct and evaluate these "objects" leading to a meanwhile richly differentiated spectrum of learning algorithms and their characterization. Robotics exerts an increasing impact on this science by introducing a number of challenges that tended to by sidestepped in earlier machine learning work that circled mainly around the idea of "mappings" or probability densities to be estimated from large numbers of passively observed examples. First of all, robots with their closed sensory-motor loops require to consider learning when the data are not only observed but also changed or even created during acting, which necessitates a perspective shift from mappings to controllers or dynamical systems. Moreover, even our simplest daily actions connect different levels of learning that seem to coexist in real-world "cognitive" learners with their embarrassingly strong generalization abilities that enable them to learn from much fewer examples than most current machine learning algorithms would require. Finally, the grounding of robots in the real world that results from their sensors, actors and their body shape, is far from trivial and has a strong impact on what can be learnt and what needs to be learnt. These and more factors make the interface between robotics and the fields of machine and of cognitive learning a rich area with exciting developments, scientific challenges and opportunities for research. The tutorial attempts a (highly selective) "tour" through some of the major issues with a discussion of the roles and lessons from robotics as a "connecting science" between the strands of machine learning and cognitive learning research. Along this way, we show and discuss examples from pertinent current work, including a status report and outlook on the recently launched project FAMULA that connects a number of CITEC research groups for developing a robot that employs manual action and language to familiarize itself with novel objects.
12:30	Lunch break
14:30	Theme 1: Learning representations: Matthias Bethge (University of Tübingen, MPI for Biological Cybernetics, Bernstein Center for Computational Neuroscience, Germany) Natural image statistics & neural representation learning An important motivation for studying the statistics of natural images is the search for image representations which facilitate visual inference tasks. Representations optimized directly for a given task are at risk of overfitting, that is, the representations might work well for that particular task but might not generalize well to others. However, the striking ability of our visual system to perform well in a variety of different situations and to recognize objects even when they have been seen only once suggests that it exploits general structural regularities of natural images. In this lecture, I will give an overview on natural image statistics and how different types of representations have been derived by modeling different statistical properties of natural images. Video Slides
16:00	Coffee break
16:30	Concluding discussion

Participants

Laith Alkurdi

Technische Universität München (Muenchen), Germany

Shun-ichi Amari

RIKEN, Japan

Tamim Asfour

KIT Karlsruhe, Germany

Nihat Ay

Max Planck Institute for Mathematics in the Sciences (Leipzig), Germany

Michael Beetz

Bremen University, Germany

Fabien Benureau

Inria Bordeaux (Talence), France

Oswald Berthold

Humboldt-Universität zu Berlin, Germany

Matthias Bethge

University of Tübingen, MPI for Biological Cybernetics, Bernstein Center for Computational Neuroscience, Germany

Dieter Büchler

Max Planck Institute for Intelligent Systems/TU Darmstadt, Germany

Christos Dimitrakakis

Chalmers University of Technology (Gothenburg), Sweden

Damien Drix

BCCN / Humboldt Universität zu Berlin, Germany

Peter Englert

Universität Stuttgart, Germany

Yanwei Fu

Queen Mary University of London, United Kingdom

Tim Genewein

Max Planck Institute for Biological Cybernetics (Tübingen), Germany

Keyan Ghazi-Zahedi

Max Planck Institute for Mathematics in the Sciences, Germany

Simon Hangl

University of Innsbruck (Pfunds), Austria

Raphael Holca-Lamarre

Technical University of Berlin, Germany

Federica Inderst

Università degli Studi di Roma 3 (Rome), Italy

Marika Kaden

UAS Mittweida, Germany

Martin Knopp

Technische Universität München, Germany

Okan Koç

MPI Tuebingen, Germany

Robin Lamarche-Perrin

Max Planck Institute for Mathematics in the Sciences (Leipzig), Germany

Marion Lange

Stuttgart University / TU Berlin (Stuttgart / Berlin), Germany

Robert Lieck

Universität Stuttgart, Germany

Qi Liu

University of Kaiserslautern, Germany

Tamas Madl

University of Manchester, United Kingdom

Guido Manfredi

LAAS-CNRS (Toulouse), France

Dimitrije Markovic

Max Planck Institute for Human Cognitive and Brain Sciences (Leipzig), Germany

Thomas Martinetz

University of Lübeck, Germany

Carlos Mastalli

Istituto Italiano di Tecnologia (Genoa), Italy

Dominik Meyer

Technische Universität München, Germany

Clément Moulin-Frier

INRIA Bordeaux Sud-Ouest, France

Hung Ngo

Swiss AI Lab IDSIA, University of Lugano (Manno-Lugano), Switzerland

Benjamin Paassen

CITEC - Bielefeld University, Germany

Paolo Perrone

Max Planck Institute for Mathematics in the Sciences (Leipzig), Germany

Anja Philippsen

Bielefeld University, Germany

Alexander Priamikov

Frankfurt Institute for Advanced Studies (Frankfurt am Main), Germany

Alexander Rietzler

University of Innsbruck, Austria

Helge Ritter

Bielefeld University, Germany

Hans-Christian Ruiz

Radboud University (Nijmegen), Netherlands

Christopher Schindlbeck

Leibniz Universität Hannover, Germany

Henry Schütze

Universität zu Lübeck, Germany

Satinder Singh

University of Michigan, USA

Sacha Sokoloski

Max Planck Institute for Mathematics in the Sciences (Leipzig), Germany

Friedrich Sommer

Redwood Center for Theoretical Neuroscience, UC Berkeley, USA

Sebastian Stabinger

University Innsbruck, Austria

Marc Toussaint

Stuttgart University, Germany

Christian Wirth

TU Darmstadt, Germany

Noga Zaslavsky

The Hebrew University of Jerusalem, Israel

Adrian Łańcucki

University of Wroclaw, Poland

Scientific Organizers

Nihat Ay

Max Planck Institute for Mathematics in the Sciences (Leipzig), Germany

Marc Toussaint

Stuttgart University, Germany

Administrative Contact

Marion Lange

Stuttgart University / TU Berlin, Germany Contact by email