Characterizing The Role of A Single Coupling Layer in Affine Normalizing Flows

Deep Affine Normalizing Flows are efficient and powerful models for high-dimensional density estimation and sample generation. Yet little is known about how they succeed in approximating complex distributions, given the seemingly limited expressiveness of individual affine layers.

In this talk, I will present the framework of Normalizing Flows for density estimation and show several recent applications like inverse problems and generative classification. Then, we take a step towards theoretical understanding by analyzing the behaviour of a single affine coupling layer under maximum likelihood loss. Such a layer estimates and normalizes conditional moments of the data distribution. One can derive a tight lower bound on the loss depending on the orthogonal transformation of the data before the affine coupling. This bound can be used to identify the optimal orthogonal transform, yielding a layer-wise training algorithm for deep affine flows.