Hybrid control laws from convex dynamic programming

In our previous paper (1999), we showed how classical ideas for dynamic programming in discrete networks can be adapted to hybrid systems. The approach is based on discretization of the continuous Bellman inequality which gives a lower bound on the optimal cost. The lower bound is maximized by linear programming to get an approximation of the optimal solution. In this paper, we apply ideas from infinite-dimensional convex analysis to get an inequality which is dual to the well known Bellman inequality. The result is a linear programming problem that gives an estimate of the approximation error in the previous numerical approaches.