A role-based POMDPs approach for decentralized implicit cooperation of multiple agents

Abstract

Decentralized decision making with uncertainty is one of the fundamental challenges in multi-agent systems. Current approaches for multi-agent coordination which rely on continuous communication of team members, cannot be applied in the practical applications where loss of communication frequently occurs. For this problem, a role-based multi-agent model is presented for implicit coordination. The model utilizes the concept of role to decompose a mission into a set of single-agent partially observable Markov decision process (POMDPs) and a task optimal assignment. Each role-based model defined with responsibilities and rights of the role can be solved with the acceptable computational complexity. The prediction of teammates' actions is the key issue in implicit coordination, for that, an action prediction algorithm based on role-based model is proposed, which estimates the current belief state by Bayes estimation and calculates the prediction of further action by the role-based policy. After obtaining the clue of the actual action through observation, the deviation of prediction is revised by filtering the prediction set with the clue. Experimental results show the validity of the proposed approach under no communication coordination.