Decentralized and Partially Decentralized Reinforcement Learning for Distributed Combinatorial Optimization Problems

Abstract

In this paper, we describe a framework for solving computationally hard, distributed function optimization problems using reinforcement learning techniques. In particular, we model a function optimization problem as an identical payoff game played by a team of reinforcement learning agents. The team performs a stochastic search through the domain space of the parameters of the function. However, current game learning algorithms suffer from significant memory requirement, significant communication overhead and slow convergence. To alleviate these problems, we present novel decentralized and partially decentralized reinforcement learning algorithms for the team. Simulation results are presented for the NP-Hard sensor subset selection problem to show that the agents learn locally optimal parameter values and illustrate the advantages of the proposed algorithms.