Numerical solution of Hamilton Jacobi Bellmann (HJB) equations and Mean Field Games by Tree Based Tensor Networks (HT/TT)

We consider Potential Mean Field Games and are focusing on the (deterministic/stochastic) HJB. In order to compute a semi-global solution, we consider first a Lagrangian perspective which is related to dynamical programing. We consider control affine dynamical systems and quadratic cost for the control. For many high dimensional PDEs of practical interest, e.g. Backward Kolmogorov equations, HJB etc., the PDE operator cannot be easily expanded in tensor form. In this case, we propose a machine learning approach confined to the manifold of tree based tensors with fixed multi-rank.

We compare a Lagrangian approach with an Eulerian method. In the Lagrangian picture we apply policy iteration and solve the linearized HJB by integrating along trajectories, defined by the corresponding dynamical system (characteristics) for samples of initial values. From the computed point values we infer the sought value function. In the stochastic case we have many paths instead of a single trajectory. There the HJB can be reformulated by an (uncoupled) Forward Backward SDE system. The forward dynamics can be computed easily by standard Euler-Mayurana scheme. For the backward equation for the value function, we use variational interpolation (Bender et al.) by solving a regression problem in each time step. For this purpose we use e.g. tree based tensor networks, in particular MPS/TT, and/or Neural Networks. The forward backward SDE is linked with (parabolic) PDEs by a non-linear Feynman-Kac theorem. Solving regression problems by means of HT/TT tensors with good approximation of the gradients requires additional attention, and has been the technical key for a successful treatment.

Joint work with M. Oster and L. Sallandt.