Applying Reward Design Based on Payment Mechanism to Shaped-Reward DQN for Beer Game

Abstract

We focus on the application of multiagent reinforcement learning for supply chain management. The beer game is an example of a problem in supply chain management and has been studied as a cooperation problem in multiagent systems. In the previous study, a method SRDQN that is based on deep reinforcement learning and reward shaping has been applied as a solution to the beer game. In the previous study of SRDQN, a single agent in a game performs reinforcement learning considering other agents to reduce the global cost for inventories of beers. However, it is possible to employ other reward shaping techniques to improve learning stability. It can also be effective in the systems consisting of multiple agents that perform reinforcement learning. We apply a reward shaping technique based on mechanism design to SRDQN to improve the cooperative policies, and then we empirically evaluate the effectiveness of the proposed approach.