Zusammenfassung für den Vortrag am 22.04.2021 (17:00 Uhr)Math Machine Learning seminar MPI MIS + UCLA
Matus Jan Telgarsky (UIUC)
Recent advances in the analysis of the implicit bias of gradient descent on deep networks
22.04.2021, 17:00 Uhr,nur Video-Broadcast
The purpose of this talk is to highlight three recent directions in the study of implicit bias — one of the current promising approaches to trying to develop a tight generalization theory for deep networks, one interwoven with optimization. The first direction is a warm-up with purely linear predictors: here, the implicit bias perspective gives the fastest known hard-margin SVM solver! The second direction is on the early training phase with shallow networks: here, implicit bias leads to good training and testing error, with not just narrow networks but also arbitrarily large ones. The talk concludes with deep networks, providing a variety of structural lemmas which capture foundational aspects of how weights evolve for any width and sufficiently large amounts of training.
Joint work with Ziwei Ji.
Bio: Matus Telgarsky is an assistant professor at the University of Illinois, Urbana-Champaign, specializing in deep learning theory. He was fortunate to receive a PhD at UCSD under Sanjoy Dasgupta. Other highlights include: co-founding, in 2017, the Midwest ML Symposium (MMLS) with Po-Ling Loh; receiving a 2018 NSF CAREER award; organizing a Simons Insititute summer 2019 program on deep learning with Samy Bengio, Aleskander Madry, and Elchanan Mossel.