Harnessing Online and Bandit Algorithms for Two Practical Challenges

Online and bandit algorithms are powerful theoretical tools in machine learning, designed for scenarios requiring sequential decision-making with limited information. In this talk, I will explore two practical applications of these algorithms. The first application is online distribution shift adaptation, where the test data distribution changes over time. By making certain assumptions about the distribution shift, we can frame this problem within an online learning context and solve it using online algorithms. The second application involves identifying the best Large Language Model (LLM)-based method in resource-constrained settings. Due to the significant costs of time or money associated with querying LLMs, it's crucial to minimize the number of queries when identifying the best LLM-base method. We demonstrate that using multi-armed bandit algorithms combined with low-rank factorization allows us to efficiently identify the best method with far fewer queries compared to evaluating all candidate methods on all test data.