Talk
Ultra-wide Neural Network and Neural Tangent Kernel
- Simon S. Du (University of Washington)
Abstract
I will talk about the result on the equivalence between the over-parameterized neural network and a new kernel, Neural Tangent Kernel. This equivalence implies two surprising phenomena: 1) the simple algorithm gradient descent provably finds the global optimum of the highly non-convex empirical risk, and 2) the learned neural network generalizes well despite being highly over-parameterized. I will also present empirical results showing Neural Tangent Kernel is a strong predictor.
