Mathematics of Machine Learning

The lecture “Mathematics of Machine Learning” serves as an introduction to analyzing common numerical problems appearing in modern machine learning applications. We will particularly focus on supervised learning and the dynamics of stochastic gradient descent. This involves the analysis of stochastic processes in discrete time, whose behavior is closely linked to deterministic, as well as stochastic, differential equations. We derive convergence rates for the classical Robbins-Monro algorithm and its Ruppert-Polyak smoothing and analyze the effect of adding inertia (momentum) to the dynamical system. Other possible topics include (stable) central limits theorems, Multilevel Monte Carlo and reinforcement learning. While the general techniques for the asymptotic analysis of stochastic processes are also introduced, proper basic knowledge of probability theory (including martingale theory) is required.

Date and time info
Wednesday, 16-18

Keywords
Stochastic Gradient Descent, Stochastic Approximation, Ruppert-Polyak averaging, Supervised learning

Prerequisites
Basic knowledge of probability theory (including martingale theory)

Remarks and notes
This is a hybrid lecture with the possibility to participate online. It will be held at the University of Bielefeld.