Optimal node visitation in acyclic stochastic digraphs

Abstract

Given a stochastic, acyclic, connected digraph with a single source node and a control agent that repetitively traverses this graph, each time starting from the source node, we want to define a control policy that will enable this agent to visit each of the graph terminal nodes a prespecified number of times, while minimizing the expected number of the graph traversals. We first formulate this problem as a specially structured discrete time Markov decision process, and we subsequently develop an asymptotically optimal randomized policy of polynomial complexity with respect to the problem size