Developing Agents for Complete DouDizhu Game Enhanced With Concurrent and Multistage Training Methods

DouDizhu is an imperfect information game involving three players, with two different bidding and cardplay phases. The large state and action spaces of the game add to its complexity. Previous studies of DouDizhu have primarily concentrated on the cardplay phase, which is the more challenging phase. As research on the cardplay agent deepens, researchers have become interested in how to train agents for the complete DouDizhu game. However, recent studies have overlooked that a poor bidding agent, which always bids 3, hinders the training of the cardplay agent. To enhance the performance of the cardplay agent (and accordingly the complete agent), this study employs a concurrent training method, in which a bidding agent is trained together with a cardplay agent and can quickly learn a good bidding play. To overcome the training difficulty encountered when the complete agent trained with the game score target, we propose a multistage training method: in the initial stage, the complete agent aims to maximize the win rate; in subsequent stages, it gradually shifts to targeting the maximum game score. Our CT-MS3-FullDouZero+ agent achieves the highest average game score at 0.228 $\pm$ 0.060, while two competing agents, the state-of-the-art CoG23+PerfectDou and ST-FullDouZero+, recorded only $-$0.002 $\pm$ 0.058 and $-$0.226 $\pm$ 0.056, respectively.