Solving Control Problems with Linear State Dynamics - A Practical User Guide

In industrial applications, practitioners usually face a considerable complexity when optimizing operating strategies under uncertainty. Typical real-world problems arising in practice are notoriously challenging from a computational viewpoint, requiring solutions to Markov Decision problems in high dimensions. In this work, we address a novel approach to obtain an approximate solution to a certain class of problems, whose state process follows a controlled linear dynamics. Our techniques is illustrated by an implementation within the statistical language R, which we discuss by solving a typical problem arising in practice.