Learning with few labeled data

The human visual system is proof that it is possible to learn new categories with very few samples; humans do not need a million samples to learn to distinguish a poisonous mushroom from an edible one in the wild. Such ability, arguably, comes from having seen millions of other categories and transferring learnt representations to the new categories. This talk will present a formal connection of machine learning with thermodynamics to characterize the quality of learnt representations for transfer learning. We will discuss how information-theoretic functionals such as rate, distortion and classification loss lie on a convex, so-called, equilibrium surface. We prescribe dynamical processes to traverse this surface under constraints, e.g., an iso-classification process that modulates rate and distortion to keep the classification loss unchanged. We demonstrate how such processes allow complete control over the transfer from a source dataset to a target dataset and can guarantee the performance of the final model. We will also discuss information-geometric methods to characterize the distance between learning tasks. This talk will discuss results from arxiv.org/abs/1909.02729, arxiv.org/abs/2002.12406 and arxiv.org/abs/2011.00613.

Bio
Pratik Chaudhari is an Assistant Professor in Electrical and Systems Engineering and Computer and Information Science at the University of Pennsylvania. He is a member of the General Robotics, Automation, Sensing and Perception (GRASP) Laboratory. From 2018-19, he was a Senior Applied Scientist at Amazon Web Services and a post-doctoral scholar in Computing and Mathematical Sciences at the California Institute of Technology. Pratik received his PhD (2018) in Computer Science from the University of California Los Angeles, his Master's (2012) and Engineer's (2014) degrees in Aeronautics and Astronautics from the Massachusetts Institute of Technology and his Bachelor’s degree (2010) from the Indian Institute of Technology Bombay. He was a part of NuTonomy Inc. (now Hyundai-Aptiv) from 2014-16 where he worked on urban autonomous navigation.