Distributed neural control for Markov decision processes in hierarchic communication networks

Abstract

We consider the problem of traffic control in a communication network hierarchically organized in order to handle different flow streams. Such a problem exhibits a partitioned control structure and is usually solved using preplanned routing mechanisms for slow varying flows. Here, we consider a Markov decision process modelling of the entities requiring admission and routing through the network under a setting allowing the slotted processing, extending a previous research (Murgu et al., 1994) done for a queueing system modelled as Brownian control problem. Using an idea of certainty equivalence principle, we map the local control scheme into a recurrent adaptive algorithm in which a learning mechanism is included. Finally, a numerical experiment with medium sized network is considered and the main conclusions are reported.<>