Cooperative Navigation via Relational Graphs and State Abstraction

Abstract

We consider a cooperative-navigation problem in a partially observable MADRL framework. We investigate how agents cooperate to learn a communication protocol given a very large state space while generalizing to a new environment. The proposed solution leverages the notion of structured observation and abstraction, in which the raw-pixel observations are converted into a relational graph that is then used for learning abstraction. Abstraction is performed based on compression using a relational graph autoencoder (RGAE) and a multilayer perceptron (MLP) to remove irrelevant information. The results show the effectiveness of the proposed MLP and RGAE in learning better policies with better generalization capabilities. It is also shown that communication among agents is instrumental in improving the navigation task performance.