Zusammenfassung für den Vortrag am 21.07.2022 (17:00 Uhr)

Math Machine Learning seminar MPI MIS + UCLA

Mark Schmidt (University of British Columbia)
Optimization Algorithms for Training Over-Parameterized Models
Siehe auch das Video dieses Vortrages.
Siehe auch die Vortragsfolien dieses Vortrages.

Over-parameterized machine learning models lead to excellent performance in a variety of applications. In this talk we consider the effect of over-parameterization on stochastic optimization algorithms. We discuss how over-parameterization allows us to use a constant step size within stochastic gradient methods, and that this leads to a faster convergence rate. We also present algorithms with provably-faster convergence rates in the over-parameterized setting. Finally, we discuss how over-parameterization allows us to update the learning rate during the training procedure which leads to improved performance over a variety of previous approaches.


02.12.2022, 10:37