Talk
Geometry and Optimization of Shallow Polynomial Networks
- Joe Kileel (UT Austin)
Abstract
In this talk I will discuss shallow neural networks with polynomial activations, with the viewpoint that understanding these networks can shed light on the behavior of more complex neural networks. I will focus on the relationship between width and optimization, as well as the effect that the data distribution has on the training landscape. Since the function space for shallow polynomial networks can be identified with symmetric tensors with bounded rank, our results may also be formulated in terms of symmetric tensor decomposition. Joint with Y. Arjevani, J. Bruna, E. Polak and M. Trager (https://arxiv.org/abs/2501.06074).
