Upcoming Events of This Seminar

Previous Events

18.04.2024 Jörg Lücke (University of Oldenburg)
On the Convergence and Transformation of the ELBO Objective to Entropy Sums
11.04.2024 Semih Cayci (RWTH Aachen)
A Non-Asymptotic Analysis of Gradient Descent for Recurrent Neural Networks
04.04.2024 Joachim Bona-Pellissier (MaLGa Universita di Genova)
Geometry-induced Implicit Regularization in Deep ReLU Neural Networks
21.03.2024 Hossein Taheri (UCSB)
Generalization and Stability in Interpolating Neural Networks
14.03.2024 Courtney Paquette (McGill University)
Hitting the High-D(imensional) Notes: An ODE for SGD learning dynamics in high-dimensions
07.03.2024 Noam Razin (Tel Aviv University)
Analyses of Policy Gradient for Language Model Finetuning and Optimal Control
22.02.2024 Tim Rudner (New York University)
Data-Driven Priors for Trustworthy Machine Learning
15.02.2024 Maksym Andriushchenko (EPFL)
A Modern Look at the Relationship between Sharpness and Generalization
08.02.2024 Julius Berner (Caltech)
Learning ReLU networks to high uniform accuracy is intractable
01.02.2024 Yulia Alexandr (University of Califorina, Berkeley)
Maximum information divergence from linear and toric models
25.01.2024 Aaron Mishkin (Stanford University)
Optimal Sets and Solution Paths of ReLU Networks
18.01.2024 Agustinus Kristiadi (Vector Institute)
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization
07.12.2023 Molei Tao (Georgia Institute of Technology)
Implicit biases of large learning rates in machine learning
30.11.2023 Pan Li (Georgia Tech)
The Stability of Positional Encoding for Graph Neural Networks and Graph Transformers
16.11.2023 Sjoerd Dirksen (Utrecht University)
The separation capacity of random neural networks
09.11.2023 Courtney Paquette (McGill University)
Hitting the High-D(imensional) Notes: An ODE for SGD learning dynamics in high-dimensions
Attention: this talk has to be cancelled
02.11.2023 Oscar Leong (Caltech)
Theory and Algorithms for Deep Generative Networks in Inverse Problems
19.10.2023 Daniel Murfet (University of Melbourne)
Phase transitions in learning machines
12.10.2023 Micah Goldblum (NYU)
Bridging the gap between deep learning theory and practice
05.10.2023 Michael Murray (UCLA)
Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?
21.09.2023 Florian Schaefer (Georgia Tech)
Solvers, Models, Learners: Statistical Inspiration for Scientific Computing
14.09.2023 Vidya Muthukumar (Georgia Institute of Technology)
Classification versus regression in overparameterized regimes: Does the loss function matter?
07.09.2023 Rahul Parhi (EPFL)
Deep Learning Meets Sparse Regularization
31.08.2023 Tom Tirer (Bar-Ilan University)
Exploring Deep Neural Collapse via Extended and Penalized Unconstrained Features Models
17.08.2023 Surbhi Goel (University of Pennsylvania)
Beyond Worst-case Sequential Prediction: Adversarial Robustness via Abstention
03.08.2023 Haoyuan Sun (MIT)
A Unified Approach to Controlling Implicit Regularization via Mirror Descent
27.07.2023 Sanghamitra Dutta (University of Maryland)
Foundations of Reliable and Lawful Machine Learning: Approaches Using Information Theory
20.07.2023 Sitan Chen (Harvard University)
Theory for Diffusion Models
13.07.2023 Behrooz Tahmasebi (MIT)
Sample Complexity Gain from Invariances: Kernel Regression, Wasserstein Distance, and Density Estimation
06.07.2023 Ronen Basri (Weizmann Institute of Science)
On the Connection between Deep Neural Networks and Kernel Methods
29.06.2023 Bingbin Liu (Carnegie Mellon University)
Thinking Fast with Transformers: Algorithmic Reasoning via Shortcuts
15.06.2023 Benjamin Bowman (UCLA and AWS)
On the Spectral Bias of Neural Networks in the Neural Tangent Kernel Regime
08.06.2023 Simone Bombari (IST Austria)
Optimization, Robustness and Privacy. A Story through the Lens of Concentration
01.06.2023 Rishi Sonthalia (UCLA)
Least Squares Denoising: Non-IID Data, Transfer Learning and Under-parameterized Double Descent
25.05.2023 Łukasz Dębowski (IPI PAN)
A Simplistic Model of Neural Scaling Laws: Multiperiodic Santa Fe Processes
18.05.2023 Duc N.M Hoang (University of Texas, Austin)
Revisiting Pruning as Initialization through the Lens of Ramanujan Graph
11.05.2023 Henning Petzka (RWTH Aachen University)
Relative flatness and generalization
27.04.2023 Stéphane d'Ascoli (ENS and FAIR Paris)
Double descent: insights from the random feature model
20.04.2023 Aditya Grover (UCLA)
Generative Decision Making Under Uncertainty
13.04.2023 Wei Zhu (University of Massachusetts Amherst)
Implicit Bias of Linear Equivariant Steerable Networks
06.04.2023 Jaehoon Lee (Google Brain)
Exploring Infinite-Width Limit of Deep Neural Networks
23.03.2023 Dohyun Kwon (University of Wisconsin-Madison)
Convergence of score-based generative modeling
16.03.2023 Chulhee Yun (KAIST)
Shuffling-based stochastic optimization methods: bridging the theory-practice gap
09.03.2023 Ricardo S. Baptista (Caltech)
On the representation and learning of triangular transport maps
02.03.2023 Samy Wu Fung (Colorado School of Mines)
Using Hamilton Jacobi PDEs in Optimization
23.02.2023 Jalaj Bhandari (Columbia University)
Global guarantees for policy gradient methods.
16.02.2023 Julia Elisenda Grigsby (Boston College)
Functional dimension of ReLU Networks
09.02.2023 Renjie Liao (University of British Columbia)
Gaussian-Bernoulli RBMs Without Tears
02.02.2023 Mihaela Rosca (DeepMind)
On continuous time models of gradient descent and instability in deep learning
26.01.2023 Liu Ziyin (University of Tokyo)
The Probabilistic Stability and Low-Rank Bias of SGD
19.01.2023 Xiaowu Dai (UCLA)
Kernel Ordinary Differential Equations
12.01.2023 Felipe Suárez-Colmenares (MIT)
Learning threshold neurons via the "edge of stability”
16.12.2022 Jona Lelmi (University of Bonn)
Large data limit of the MBO scheme for data clustering
15.12.2022 Nicolás García Trillos (University of Wisconsin-Madison)
Analysis of adversarial robustness and of other problems in modern machine learning
08.12.2022 Mojtaba Sahraee-Ardakan (UCLA)
Equivalence of Kernel Methods and Linear Models in High Dimensions
01.12.2022 Levon Nurbekyan (UCLA)
Efficient natural gradient method for large-scale optimization problems
17.11.2022 Sebastian Goldt (SISSA Trieste)
On the importance of higher-order input statistics for learning in neural networks
10.11.2022 El Mehdi Achour (IMT Toulouse)
The loss landscape of deep linear neural networks: A second-order analysis
03.11.2022 Itay Safran (Purdue University)
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias
27.10.2022 Lechao Xiao (Google Brain)
Eigenspace restructuring: A Principle of space and frequency in neural networks
20.10.2022 Joe Kileel (University of Texas at Austin)
The Effect of Parametrization on Nonconvex Optimization Landscapes
13.10.2022 Chao Ma (Stanford University)
Implicit bias of optimization algorithms for neural networks: static and dynamic perspectives
06.10.2022 Felix Dietrich (Technical University of Munich)
Learning dynamical systems from data
22.09.2022 Gintare Karolina Dziugaite (Google Brain)
Deep Learning through the Lens of Data
15.09.2022 Lisa Maria Kreusser (University of Bath)
Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)
08.09.2022 Devansh Arpit (Salesforce)
Optimization Aspects that Improve IID and OOD Generalization in Deep Learning
25.08.2022 Melanie Weber (Harvard University)
Exploiting geometric structure in (matrix-valued) optimization
18.08.2022 Aditya Golatkar (UCLA)
Selective Forgetting and Mixed Privacy in Deep Networks
12.08.2022 Francesco Di Giovanni (Twitter)
Over-squashing and over-smoothing through the lenses of curvature and multi-particle dynamics
04.08.2022 Wei Hu (University of California, Berkeley)
More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize
28.07.2022 Arash A. Amini (UCLA)
Target alignment in truncated kernel ridge regression
21.07.2022 Mark Schmidt (University of British Columbia)
Optimization Algorithms for Training Over-Parameterized Models
14.07.2022 Rong Ge (Duke University)
Towards Understanding Training Dynamics for Mildly Overparametrized Models
07.07.2022 Sebastian Kassing (University of Bielefeld)
Convergence of Stochastic Gradient Descent for analytic target functions
23.06.2022 Lisa Maria Kreusser (University of Bath)
Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)
ATTENTION - this talk has been cancelled.
16.06.2022 Jingfeng Wu (Johns Hopkins University)
A Fine-Grained Characterization for the Implicit Bias of SGD in Least Square Problems
09.06.2022 Guang Cheng (UCLA)
Nonparametric Perspective on Deep Learning
02.06.2022 Tan Nguyen (UCLA)
Principled Models for Machine Learning
26.05.2022 Melikasadat Emami (UCLA)
Asymptotics of Learning in Generalized Linear Models and Recurrent Neural Networks
19.05.2022 Sayan Mukherjee (MPI MiS, Leipzig + Universität Leipzig)
Inference in dynamical systems and the geometry of learning group actions
12.05.2022 Zixiang Chen (UCLA)
Benign Overfitting in Two-layer Convolutional Neural Networks
05.05.2022 Noam Razin (Tel Aviv University)
Generalization in Deep Learning Through the Lens of Implicit Rank Minimization
21.04.2022 Berfin Şimşek (Ecole Polytechnique Fédérale de Lausanne (EPFL))
The Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetry-Induced Saddles and Global Minima Manifold
14.04.2022 Christoph Lampert (Institute of Science and Technology, Austria)
Robust and Fair Multisource Learning
07.04.2022 Chaoyue Liu (Ohio State University)
Transition to Linearity of Wide Neural Networks
31.03.2022 Benjamin Eysenbach (Carnegie Mellon University & Google Brain)
The Information Geometry of Unsupervised Reinforcement Learning
24.03.2022 Ohad Shamir (Weizmann Institute of Science)
Implicit bias in machine learning
17.03.2022 Gabin Maxime Nguegnang (RWTH Aachen University)
Convergence of gradient descent for learning deep linear neural networks
11.03.2022 Uri Alon (CMU)
Bottlenecks, over-squashing, and attention in Graph Neural Networks.
10.03.2022 Gergely Neu (Universitat Pompeu Fabra, Barcelona)
Generalization Bounds via Convex Analysis
04.03.2022 Daniel McKenzie (UCLA)
Implicit Neural Networks: What they are and how to use them.
03.03.2022 Marco Mondelli (Institute of Science and Technology Austria)
Gradient Descent for Deep Neural Networks: New Perspectives from Mean-field and NTK
24.02.2022 Danijar Hafner (Google Brain & University of Toronto)
General Infomax Agents through World Models
17.02.2022 Maksim Velikanov (Skolkovo Institute of Science and Technology)
Spectral properties of wide neural networks and their implications to the convergence speed of different Gradient Descent algorithms
10.02.2022 Hui Jin (University of California, Los Angeles)
Learning curves for Gaussian process regression with power-law priors and targets
04.02.2022 Song Mei (University of California, Berkeley)
The efficiency of kernel methods on structured datasets
27.01.2022 Daniel Russo (Columbia University)
Adaptivity and Confounding in Multi-armed Bandit Experiments
20.01.2022 Philipp Petersen (University of Vienna)
Optimal learning in high-dimensional classification problems
13.01.2022 Yuan Cao (University of California, LA)
Towards Understanding and Advancing the Generalization of Adam in Deep Learning
16.12.2021 Zhi-Qin John Xu (Shanghai Jiao Tong University)
Occam’s razors in neural networks: frequency principle in training and embedding principle of loss landscape
02.12.2021 Ekaterina Lobacheva (HSE University), Maxim Kodryan (HSE University)
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay
25.11.2021 Michael Joswig (TU Berlin)
Geometric Disentanglement by Random Convex Polytopes
18.11.2021 Daniel Soudry (Technion)
Algorithmic Bias Control in Deep learning
11.11.2021 Christoph Hertrich (TU Berlin)
Towards Lower Bounds on the Depth of ReLU Neural Networks
04.11.2021 Hyeyoung Park (Kyungpook National University)
Effect of Geometrical Singularity on Learning Dynamics of Neural Networks
21.10.2021 David Stutz (MPI for Informatics, Saarbrücken)
Relating Adversarial Robustness and Weight Robustness Through Flatness
14.10.2021 Mathias Drton (Technical University of Munich)
A Bayesian information criterion for singular models
07.10.2021 Roger Grosse (University of Toronto)
Self-tuning networks: Amortizing the hypergradient computation for hyperparameter optimization
23.09.2021 Huan Xiong (MBZUAI, Abu Dhabi)
On the Number of Linear Regions of Convolutional Neural Networks with Piecewise Linear Activations
16.09.2021 Pavel Izmailov (New York University)
What Are Bayesian Neural Network Posteriors Really Like?
09.09.2021 Shaowei Lin (working in a stealth startup)
All you need is relative information
02.09.2021 Matthew Trager (Amazon US)
A Geometric View of Functional Spaces of Neural Networks
26.08.2021 Guodong Zhang (University of Toronto)
Differentiable Game Dynamics: Hardness and Complexity of Equilibrium Learning
22.07.2021 Soledad Villar (Johns Hopkins University)
Scalars are universal: Gauge-equivariant machine learning, structured like classical physics
15.07.2021 Hossein Mobahi (Google Research)
Self-Distillation Amplifies Regularization in Hilbert Space
08.07.2021 Roger Grosse (University of Toronto)
Self-tuning networks: Amortizing the hypergradient computation for hyperparameter optimization
01.07.2021 Dominik Janzing (Amazon Research, Tuebingen, Germany)
Why we should prefer simple causal models
17.06.2021 Ziv Goldfeld (Cornell University)
Scaling Wasserstein distances to high dimensions via smoothing
10.06.2021 Isabel Valera (Saarland University and MPI for Intelligent Systems)
Algorithmic recourse: theory and practice
03.06.2021 Stanisław Jastrzębski ( and Jagiellonian University)
Reverse-engineering implicit regularization due to large learning rates in deep learning
27.05.2021 Sebastian Reich (University of Potsdam and University of Reading)
Statistical inverse problems and affine-invariant gradient flow structures in the space of probability measures
13.05.2021 Eirikur Agustsson (Google research)
Universally Quantized Neural Compression
06.05.2021 Haizhao Yang (Purdue University)
A Few Thoughts on Deep Learning-Based PDE Solvers
29.04.2021 Mehrdad Farajtabar (DeepMind)
Catastrophic Forgetting in Continual Learning of Neural Networks
22.04.2021 Matus Jan Telgarsky (UIUC)
Recent advances in the analysis of the implicit bias of gradient descent on deep networks
15.04.2021 Stanislav Fort (Stanford University)
Neural Network Loss Landscapes in High Dimensions: Theory Meets Practice
08.04.2021 Cristian Bodnar (University of Cambridge), Fabrizio Frasca (Twitter)
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks
01.04.2021 Roi Livni (Tel Aviv University)
Regularization, what is it good for?
25.03.2021 Pratik Chaudhari (University of Pennsylvania)
Learning with few labeled data
18.03.2021 Ryo Karakida (AIST (National Institute of Advanced Industrial Science and Technology), Tokyo)
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
11.03.2021 Spencer Frei (Department of Statistics, UCLA)
Generalization of SGD-trained neural networks of any width in the presence of adversarial label noise