Geometry plays a crucial role in many Machine Learning problems, as it effectively captures the underlying structure and regularity of data. Additionally, Geometry provides a versatile framework for analyzing, unifying and generalizing Machine Learning methods to new settings.

GaML-23 aims to foster interaction and collaboration among researchers and practitioners working at the intersection of Geometry and Machine Learning. The Workshop will consist of invited talks, including keynotes by well-known experts and hands-on tutorials on implementing Geometric Machine Learning algorithms.

Limited funds are available to cover some travel and accommodation costs for early-career participants, mainly targeting postdoctoral researchers and Ph.D. students. Applicants are expected to submit a brief academic CV and a statement of purpose, Ph.D. students applying for funding must additionally provide a letter of support from advisor(s). The funding application is included in the registration form.

Deadlines

Applications for funding requests are closed now.

In-person registration for the Main Venue (MPI MiS, E1 05) has reached capacity and is now closed. It is still possible to register for the live stream.

WARNING: Please be wary if a company directly contacts you regarding payment for accommodation in Leipzig: this is fraud / phishing.

In many applications, especially in the sciences, data and tasks have known invariances. Encoding such invariances directly into a machine learning model can improve learning outcomes, while it also poses challenges on efficient model design.
In the first part of the talk, we will focus on the invariances relevant to eigenvectors and eigenspaces being inputs to a neural network. Such inputs are important, for instance, for graph representation learning. We will discuss targeted architectures that can universally express functions with the relevant invariances or equivariances - sign flips and changes of basis - and their theoretical and empirical benefits.Second, we will take a broader, theoretical perspective. Empirically, it is known that encoding invariances into the machine learning model can reduce sample complexity. For the simplified setting of kernel ridge regression or random features, we will discuss new bounds that illustrate two ways in which invariances can reduce sample complexity. Our results hold for learning on manifolds and for invariances to (almost) any group action.This talk is based on joint work with Joshua Robinson, Derek Lim, Behrooz Tahmasebi, Lingxiao Zhao, Tess Smidt, Suvrit Sra and Haggai Maron.Bio: Stefanie Jegelka is a Humboldt Professor at TU Munich and and Associate Professor (on leave) in the Department of EECS at MIT. Before joining MIT, she was a postdoctoral researcher at UC Berkeley, and obtained her PhD from ETH Zurich and the Max Planck Institute for Intelligent Systems. Stefanie has received a Sloan Research Fellowship, an NSF CAREER Award, a DARPA Young Faculty Award, the German Pattern Recognition Award, a Best Paper Award at ICML and an invitation as sectional lecturer at the International Congress of Mathematicians. She has co-organized multiple workshops on (discrete) optimization in machine learning and graph representation learning, and has served as an Action Editor at JMLR and a program chair of the International Conference on Machine Learning (ICML) 2022. Her research interests span the theory and practice of algorithmic machine learning, in particular, learning problems that involve combinatorial, algebraic or geometric structure.

Artificial neural networks are for example prone to being fooled by carefully perturbed inputs which cause an egregious misclassification. Graph theory has been extensively used to model real data such as brain, social interactions and others. In this talk, I will show how graphs-based approaches help to investigate the inner workings of artificial neural networks in two different experiments: adversarial attacks and catastrophic forgetting.

We will consider modeling shapes and fields via topological and lifted-topological transforms. Specifically, we show how the Euler Characteristic Transform and the Lifted Euler Characteristic Transform can be used in practice for statistical analysis of shape and field data. We also state a moduli space of shapes for which we can provide a complexity metric for the shapes. We also provide a sheaf theoretic construction of shape space that does not require diffeomorphisms or correspondence. A direct result of this sheaf theoretic construction is that in three dimensions for meshes, 0-dimensional homology is enough to characterize the shape. We will also discuss Gaussian processes on fiber bundles and applications to evolutionary questions about shapes. Applications in biomedical imaging and evolutionary anthropology will be stated throughout the talk.

AI systems often need to process data that finds its origins in our real physical world, and is thus inherently grounded in geometry and physics. It is then evident that neural architectures capable of preserving this grounding in their representations excel in tasks involving critical geometric aspects. This observation is particularly noticeable in domains such as computational chemistry, physics, and fields like medical image analysis.
In this talk, I will argue that equivariant neural networks are precisely the class of architectures that excel in preserving this notion of geometric and physical grounding. I will present the idea that these networks can be viewed as creating neural ideograms —geometric symbols that can represent both literal geometric objects (pictograms) and more abstract concepts (ideograms).
Furthermore, I will showcase specific examples of equivariant methods that employ geometric objects (such as multi-vectors and spherical harmonics/irreps) as internal hidden states of neurons, in contrast to the conventional use of standard Euclidean vectors. I will argue that the use of such internal states for representing geometric information allows neural networks to excel in tasks demanding a form of geometric reasoning. To illustrate this, I will show an application of these techniques in the generative modelling of molecules.
As a technical contribution, in addition to the more conceptual ideas mentioned above, I will present a simple recipe to build equivariant architectures based on a generalized definition of weight sharing as conditional message passing, with interaction functions conditioned on pair-wise invariant attributes that represent equivalence classes of point-pairs.

One of the key reasons for the success of geometric deep learning is that it allows a significant dimensionality reduction thanks to the introduction of an inductive bias based on prior knowledge of the symmetries of the data studied. Parallel to the development of geometric deep learning techniques, the theory of group equivariant non-expansive operators (GENEO) has taken shape. GENEOs give a topological model to encode equivariance and can be used to design layers of neural networks. After introducing GENEOs and some applications related to them, we will discuss a generalisation to partial equivariance. The main reason for this is that, due to noise and incompleteness of data, the symmetries of a data set cannot always be modelled as a group. Thus, we need to generalise GENEOs and allow for partial equivariance to be taken into account. The space of partially equivariant non-expansive operators (P-GENEOs) maintains most of the nice features of the space of GENEO, such as compactness and convexity.

Conditional Gradient methods are an important class of methods to minimize (non-)smooth convex functions over (combinatorial) polytopes. Recently these methods received a lot of attention as they allow for structured optimization and hence learning, incorporating the underlying polyhedral structure into solutions. In this talk I will give a broad overview of these methods, their applications, as well as present some recent results both in traditional optimization and learning as well as in deep learning.

We study the loss landscape of two-layer mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss. Our approach involves bounding the dimension of the sets of local and global minima using the rank of the Jacobian of the parametrization map. Using results on random binary matrices, we show most activation patterns correspond to parameter regions with no bad differentiable local minima. Furthermore, for one-dimensional input data, we show most activation regions realizable by the network contain a high dimensional set of global minima and no bad local minima. We experimentally confirm these results by finding a phase transition from most regions having full rank to many regions having deficient rank depending on the amount of overparameterization.
This is work with Kedar Karhadkar, Michael Murray, Hanna Tseran.
Bio: Guido Montúfar is an Associate Professor of Mathematics and Statistics & Data Science at UCLA as well as the Leader of the Math Machine Learning Group at the Max Planck Institute for Mathematics in the Sciences. His research focuses on deep learning theory and more generally mathematical aspects of machine learning. He studied mathematics and physics at TU Berlin, obtained the Dr.rer.nat. in 2012 as an IMPRS fellow in Leipzig, and held postdoc positions at PennState and MPI MiS. Guido Montufar is a 2022 Alfred P. Sloan Research Fellow.
https://www.math.ucla.edu/~montufar/

Random walks on discrete settings is one of the most powerful mathematical tools for solving a wide range of theoretical and applied problems in discrete math, probability, theoretical computer science, data science, and machine learning. This ranges from google page rank algorithm to clustering elements in complex networks and detecting topological communities in high dimensional data. In this talk, I will show how random walks can serve as the bridge between quantitative features of data, which are determined by geometric tools such as curvature, and qualitative features that can be detected by topological methods. In traditional data analysis, graphs as the simplest examples of discrete structures are well studied and we have a relatively broad understanding of their topological, geometric and stochastic properties. However, graphs can not present higher order interactions among data points and a main challenge in both theoretical and applied sides is to extend results from graphs to more complicated structures which are suitable for modelling beyond binary interactions. After presenting some of the main ideas of geometric and topological data analysis via random walks on graphs, I will talk about the main challenges of extending some graph-results to higher dimensions and how we can go further. I try to present mostly the intuition behind the ideas and will not go into all the details. Therefore I hope the content would be accessible to those who have not much background in geometric or topological methods. Knowing about Markov chains and stochastic processes would be helpful.

Graph neural networks have two different data modes - the feature vectors and the graph structure. In this talk, we explore the interaction between hyperbolic geometry and the two different data formats.
For the feature vectors, we explore the effect of geometry on the performance of three types of GNNs for node classification and link prediction. To do so, we propose a family of mixed geometry GNNs with Hyperbolic and Euclidean components that can be trained jointly. We compare the performance of our mixed geometry models against their Euclidean and Hyperbolic counterparts across various datasets. We see that the mixed geometry models have the best performance for node classification, while the Hyperbolic models have the best performance for link prediction. Further, we see that for node classification, the choice of architecture had a more significant impact on the performance than the choice of geometry. Whereas for link prediction, the choice of geometry had a much more significant impact than the choice of architecture.
For the graph structure, we study the problem of oversquashing. We use ideas from geometric group theory to present RelWire, a rewiring technique based on the geometry of the graph. We derive topological connections for RelWire. We then rewire different real-world molecule datasets and show that RelWire is Pareto optimal: it has the best balance between improvement in eigengap and commute times and minimizing changes in the topology of the underlying graph.

The set of functions parameterized by a linear fully-connected neural network is a determinantal variety. We investigate the subvariety of functions that are equivariant or invariant under the action of a permutation group. Examples of such group actions are translations or 90◦ rotations on images. For such equivariant or invariant subvarieties, we provide an explicit description of their dimension, their degree as well as their Euclidean distance degree, and their singularities. We fully characterize invariance for arbitrary permutation groups, and equivariance for cyclic groups. We draw conclusions for the parameterization and the design of equivariant and invariant linear networks, such as a weight sharing property, and we prove that all invariant linear functions can be learned by linear autoencoders.
This talk is based on joint work with Anna-Laura Sattelberger and Vahid Shahverdi.

Spatial trajectories, often represented as a sequence of spatial positions, are a standard way to represent human mobility patterns. They also are used to represent motion patterns including for animals, drones, or last-mile rentals (e-scooters). However, these trajectories are notoriously difficult to work with as they overlap and can stretch long distances.
In this talk we discuss a new sketch (the minDist Sketch) that makes just about any data analysis on trajectories tasks simple and efficient. This first considers spatial trajectories as an abstract shape, and then maps them to a high-dimensional Euclidean space as a vector. We can show recovery, pseudo-metric, and metric properties of this representation. Variants can include direction information, or traits like velocity and acceleration.
Moreover, once represented as this vector, the trajectory data is extremely easy to work with. Allowing for out-of-the-box use of software for nearest-neighbor search, clustering, and classification. In particular, we conduct the first formal study of classifying spatial trajectories: given trajectories from two different distributions (e.g., generated by car or bus) given a new trajectory that is unlabeled, how well can we predict which class it was from?
Over several data sets we have assembled that demand this task, we conduct a large study, and show that the minDist sketch and its variants are consistently the easiest and most accurate method (or at the least among the best in each instance).
Joint work with Pingfan Tang and Hasan Pourmahmood

Ideally, the distribution of the latent representations within any neural network should depend only on the task, the data, the loss, and other architecture-specific constraints. However, factors such as the random weights initialization, hyperparameters, or other sources of randomness may induce incoherent latent spaces that hinder any form of reuse. In this talk I'll report an important (and somewhat surprising) empirical observation: under the same data and modeling choices, distinct latent spaces typically differ by an unknown quasi-isometric transformation; that is, in each space, the distances between the encodings do not change. I'll then show how simply adopting pairwise similarities as an alternative data representation leads to guaranteed isometry invariance of the latent spaces, effectively enabling latent space communication: from zero-shot model stitching to latent space comparison between diverse settings. Several validation experiments will follow on different datasets, spanning various modalities (images, text, graphs), tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs, GCNs, transformers).

Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces X and Y using elementary functions between Euclidean spaces as building blocks. Earlier results assume that the output space Y is a topological vector space. We overcome this limitation by “randomization”: our approximators output discrete probability measures over Y. When X and Y are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for Hölder-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of X and Y. For barycentric Y, including Banach spaces, R-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to Y-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.

The Structure from Motion (SfM) problem aims to create a tridimensional model of a scene from 2D images. In order to do so, point and line features are identified and matched in the images, creating correspondences that are triangulated and used to recreate the scene. In this talk we will focus on the triangulation of lines. We will define the line multiview varieties and present their main properties. Then we will introduce the Anchored Multiview varieties which will be a starting point for the triangulation of line and point incidences appearing frequently in line segment triangulation.

This talk will explain that Convolutional Neural Networks without activation parametrize semialgebraic sets of real homogeneous polynomials that admit a certain sparse factorization. We will investigate how the geometry of these semialgebraic sets (e.g., their singularities and relative boundary) changes with the network architecture. Moreover, we will explore how these geometric properties affect the optimization of a loss function for given training data. This talk is based on joint work with Kathlén Kohn, Guido Montúfar, and Matthew Trager.

This tutorial provides a practical demonstration of how numerical methods can enhance the comprehension of (algebraic) geometric problems and aid in proving statements. We begin with a brief overview of numerical homotopy continuation methods and the process of certifying solutions with interval methods.
Utilizing the software HomotopyContinuation.jl, we delve into practical applications, showcasing the techniques through a series of examples.

Participants

Asma Abid

Government College University

Sophie Achard

CNRS, Univ. Grenoble Alpes

Sharanya Achut

Georg-August-Universität Göttingen

Usman Ali

Bahauddin Zakariya University

Shivam Arora

Memorial University of Newfoundland

George Balla

TU Berlin

Jacob Bamberger

University of Oxford

Federico Barbero

University of Oxford

Abhishek Behera

MPI-CBG/ CSBD

Maxim Beketov

HSE University

Erik Bekkers

University of Amsterdam

Jonas Beyrer

University of Bonn

Alessio Borzì

MPI MIS Leipzig

Marie-Charlotte Brandenburg

MPI MiS Leipzig

Pierre Bréchet

MPI MiS

Paul Breiding

University of Osnabrueck

Sasha Brenner

MPI CBS

Sven Buchholz

TH Brandenburg

Evgeny Burnaev

Skoltech

Fernando Camacho Cadena

MPI MiS Leipzig

Jorge Eduardo Cardona Gaviria

Friedrich Schiller University Jena

Oscar Carlsson

Chalmers University of Technology

Jonas Cassel

Heidelberg University

Karel Devriendt

MPI MiS Leipzig

Valentina Disarlo

Heidelberg University

Nicholas Early

MPI MiS Leipzig

Marzieh Eidi

MPI MiS Leipzig/ SCaDs. AI

Yara Elshiaty

Heidelberg Univeristy

Fatemeh Fahimi

MPI MiS Leipzig

Samantha Fairchild

MPI MiS Leipzig

Ines Garcia Redondo

Imperial College London

Marina Garrote-López

MPI MiS Leipzig

Andre Gomes

Universtiy of Campinas

Daniel Gonzalez

Heidelberg University

Pablo Gottheil

Leipzig University

Vincent Grande

RWTH Aachen University

Heather Harrington

MPI-CBG

Michael Hecht

CASUS/HZDR and University Wroclaw

Shah Hussain

Graz University of Technology

Mitul Islam

MPI MiS Leipzig

Shamoona Jabeen

University of Science & Technology Bannu

Saiyam Jain

KIT

Abel Jansma

MPI MiS Leipzig

Stefanie Jegelka

Massachusetts Institute of Technology

Parvaneh Joharinad

ScaDS.AI Dresden/ MPI MiS Leipzig

Jürgen Jost

MPI MIS Leipzig

Oleg Kachan

HSE University

Leonie Kayser

MPI MIS Leipzig

Janis Keck

MPI MIS Leipzig

Martin Keller-Ressel

TU Dresden

Yash Kesharwani

Savitribai Phule Pune University

Dimitra Kiakou

MPI CBS Leipzig

Sayako Kodera

Fraunhofer Institute for Nondestructive Testing IZFP

Joris Koefler

University of Warwick

Constantin Kogler

University of Oxford

Kathlén Kohn

KTH Stockholm

Fabius Krämer

Rheinische Friedrich-Wilhelms-Universität Bonn

Anastasis Kratsios

McMaster and Vector Institute

Oxana Krymina

VSK (Insurance company)

Deniz Kucukahmetler

Imperial College London

Olga Kuznetsova

Aalto University

Fabian Lander

University of Heidelberg

Gennady Laptev

Skoltech

Kolya Lettl

Leipzig University

Jiayi Li

University of California, Los Angeles

Ruowei Li

MPI MIS Leipzig

Felix Lotter

University of Bonn

Kelly Maggs

EPFL

Marta Magnani

University of Heidelberg

Filippo Mazzoli

MPI MIS Leipzig

Madhumita Mondal

The Institute of Mathematical Sciences

Guido Montúfar

MPI MIS Leipzig/ UCLA

Paula Mühlmeyer

Carl von Ossietzky Universität Oldenburg

Sayan Mukherjee

ScaDS.AI/ MPI MiS Leipzig

Johannes Müller

RWTH Aachen

Merik Niemeyer

MPI MiS Leipzig

Elias Nyholm

Chalmers University of Technology / University of Gothenburg

John Ochen

Makerere University

Osman Okutan

MPI MiS Leipzig

Eckehard Olbrich

MPI MiS Leipzig

ANGÉLICA OLIVEIRA

UNICAMP

Arsenii Onuchin

Skoltech

Milka Anyango Otieno

Kenya Commercial Bank/Chuka University

Apondi Otieno

Technical University of Kenya

Maks Ovsjanikov

Ecole Polytechnique

Guendalina Palmirotta

University of Luxembourg

Niharika Paul

MPI MiS Leipzig

Eduardo Jose Perez Mejia

Fraunhofer-Institut für Zerstörungsfreie Prüfverfahren IZFP

Bei Wang Phillips

University of Utah

Jeff Phillips

University of Utah

Sebastian Pokutta

Zuse Institute Berlin

Renata Possobon

Unicamp/ MPI MiS Leipzig

Naeem Ahmad Pundeer

Jadavpur University, Kolkata, India

Kristian Ranestad

University of Oslo

Bernhard Reinke

MPI MiS Leipzig

Guillermo Restrepo

MPI MiS Leipzig

Emanuele Rodolà

Sapienza University of Rome

Kilian Roth

Leipzig University

Roua Rouatbi

TU Dresden

Leonel Rozo

Bosch Center for Artificial Intelligence

Elfat Sabitov

NRU HSE

Nico Scherf

MPI CBS Leipzig

Ilya Schurov

Radboud University

Vahid Shahverdi

KTH Royal Institute of Technology

Ehsan Shams

Alexandria University

Elima Shehu

Osnabrück University & MPI MiS

Jiajun Shi

MPI MiS Leipzig

Dmitrij Sitenko

University of Heidelberg

Mireille Soergel

MPI MiS Leipzig

Linus Sommer

Leipzig University

Rishi Sonthalia

UCLA

Daniel Speckhard

FHI-Berlin der MPG, HU-Berlin, MPI-FKF

Daniel Spitz

MPI MiS Leipzig

Simone Steinbrüchel

Leipzig University

Jan Stühmer

Heidelberg Institute for Theoretical Studies

Jan Stühmer

Heidelberg Institute for Theoretical Studies / Karlsruhe Institute of Technology

Dominik Sturm

TU Dresden

Bernd Sturmfels

MPI MiS Leipzig

Juan Esteban Suarez

TU Dresden

Diaaeldin Taha

MPI MiS Leipzig

Roan Talbut

Imperial College London

Sascha Timme

TU Berlin

Francesca Tombari

MPI MiS Leipzig

Nahid Torbati

MPI CBS Leipzig

Angelica Torres

MPI MiS Leipzig

Anton Ullrich

MPI MiS Leipzig

Vahideh Vahidifar

University of Sussex

Jörg Walter

Leipzig University

Qingsong Wang

University of Utah

Qiquan Wang

Imperial College London

Anna Wienhard

MPI MiS Leipzig

Arne Wolf

Imperial College London

Neza Zager Korenjak

MPI MiS Leipzig

Daniil Zaytsev

Higher School of Economics

ANUM ZEHRA

THE WOMEN UNIVERSITY MULTAN

Wei Zhao

Heidelberg Institute for Theoretical Studies

Giulio Zucal

MPI MiS Leipzig

Scientific Organizers

Samantha Fairchild

Max Planck Institute for Mathematics in the Sciences

Diaaeldin Taha

Max Planck Institute for Mathematics in the Sciences

Anna Wienhard

Max Planck Institute for Mathematics in the Sciences

Administrative Contact

Katharina Matschke

Max Planck Institute for Mathematics in the Sciences
Contact by email