Multi-agent policy transfer via task relationship modeling

Abstract

Team adaptation to new cooperative tasks is a hallmark of human intelligence, which has yet to be fully realized in learning agents. Previous studies on multi-agent transfer learning have accommodated teams of different sizes but heavily relied on the generalization ability of neural networks for adapting to unseen tasks. We posit that the relationship among tasks provides key information for policy adaptation. We utilize this relationship for efficient transfer by attempting to discover and exploit the knowledge among tasks from different teams, proposing to learn an effect-based task representation as a common latent space among tasks, and using it to build an alternatively fixed training scheme. Herein, we demonstrate that task representation can capture the relationship among teams and generalize to unseen tasks. Thus, the proposed method helps transfer the learned cooperation knowledge to new tasks after training on a few source tasks. Furthermore, the learned transferred policies help solve tasks that are difficult to learn from scratch.