Multi-Agent Reinforcement Learning Reward Engineering via Stochastic Game Evaluation

Abstract

With the proliferation of Reinforcement Learning (RL) algorithms across multiple applications, the design of appropriate reward mechanisms that elicit desired behaviours becomes crucial. Reward setting is made more difficult in the multi-agent case where cooperative, competitive or mixed interactions between agents may lead to differing outcomes. In this paper, we formulate the reward engineering of multi-agent reinforcement learning approaches via game theoretic models. This approach is used to analyze the overall team reward when choosing one game theoretic structure over another. An empirical analysis is provided over game theoretic simulators that demonstrate co-operative game rewards improve the rewards by upto 30%. This formulation may be applied to a variety of use cases within logistics, transport and telecommunications domains that employ multi-agent techniques.