Data and Model Geometry in Deep Learning

The utility of encoding geometric structure, such as known symmetries, into machine learning architectures has been demonstrated empirically in domains ranging from biology to computer vision.However, rigorous analysis of its impact on the learning complexity of neural networks remains largely unexplored. Recent advances in learning theory have shown that training shallow, fully connected neural networks, which are agnostic to data geometry, has exponential complexity in the statistical query model, a framework encompassing gradient descent. In this talk, we ask whether knowledge on data and model geometry is sufficient to alleviate the fundamental hardness of learning neural networks. We analyze two geometric settings: First, equivariant neural networks, a class of geometric architectures that explicitly encode symmetries; and second, learning neural networks under the manifold hypothesis, which assumes that high-dimensional input data lies on or near a low-dimensional manifold.