A Geometric View of Functional Spaces of Neural Networks

In this talk, I will present some of my work on the functional space associated with neural networks. I will focus on simple classes of networks, including feedforward networks with linear and polynomial activations and two-layer ReLU networks, that provide a tractable setting where many geometric properties of general networks can be studied in detail. In particular, I will emphasize the distinction between the intrinsic function space and its parameterization, in order to shed light on the impact of the architecture on the expressivity of a model and on the corresponding optimization landscapes.

Work done outside Amazon.