Topological Aspects of Symmetry-Preserving Neural Networks
- Jonathan Siegel (Texas A&M University)
Abstract
In many practical applications of machine learning, especially to scientific disciplines like physics, chemistry, or biology, the ground truth satisfies some known symmetries. Mathematically, this corresponds to invariance or equivariance of the prediction function with respect to a certain group of symmetries, typically the rotation or permutation groups. As a simple example, the chemical properties of a molecule are invariant to rotations. In such applications, it is often highly desirable to build these symmetries into the neural network model. We will analyze two different methods for doing this: constructing special architectures which preserve the desired symmetries, and building invariance into a standard (non-invariant) architecure by preprocessing the input and postprocessing the output. For the former, we will discuss universality and approximation rates for the popular permutation invariant Deep Sets architecture. For the latter, we will discuss the construction of canonicalizations and weighted frames for the actions of permutations and rotations. In both of these examples, we will see interesting topological problems which govern the limitations of these methods.
