Abstract for the talk on 06.08.2020 (17:00 h)Math Machine Learning seminar MPI MIS + UCLA
Simon S. Du (University of Washington)
Ultra-wide Neural Network and Neural Tangent Kernel
See the video of this talk.
I will talk about the result on the equivalence between the over-parameterized neural network and a new kernel, Neural Tangent Kernel. This equivalence implies two surprising phenomena: 1) the simple algorithm gradient descent provably finds the global optimum of the highly non-convex empirical risk, and 2) the learned neural network generalizes well despite being highly over-parameterized. I will also present empirical results showing Neural Tangent Kernel is a strong predictor.