Abstract for the talk on 27.08.2020 (17:00 h)
Math Machine Learning seminar MPI MIS + UCLAFelix Draxler (Heidelberg University)
Characterizing The Role of A Single Coupling Layer in Affine Normalizing Flows
See the video of this talk.
Deep Affine Normalizing Flows are efficient and powerful models for high-dimensional density estimation and sample generation. Yet little is known about how they succeed in approximating complex distributions, given the seemingly limited expressiveness of individual affine layers.
In this talk, I will present the framework of Normalizing Flows for density estimation and show several recent applications like inverse problems and generative classification. Then, we take a step towards theoretical understanding by analyzing the behaviour of a single affine coupling layer under maximum likelihood loss. Such a layer estimates and normalizes conditional moments of the data distribution. One can derive a tight lower bound on the loss depending on the orthogonal transformation of the data before the affine coupling. This bound can be used to identify the optimal orthogonal transform, yielding a layer-wise training algorithm for deep affine flows.