Data-Driven Inverse Reinforcement Learning for Heterogeneous Optimal Robust Formation Control

Abstract

This article presents novel data-driven inverse reinforcement learning (IRL) algorithms to optimally address heterogeneous formation control problems in the presence of disturbances. We propose expert-estimator-learner multiagent systems (MASs) as independent systems with similar interaction graphs. First, a model-based IRL algorithm is introduced for the estimator MAS to determine its optimal control and reward functions. Using the estimator IRL algorithm results, a robust algorithm for model-free IRL is presented to reconstruct the learner MAS’s optimal control and reward functions without knowing the learners’ dynamics. Therefore, estimator MAS aims to estimate experts’ desired formation and learner MAS wants to track the estimators’ trajectories optimally. As a final step, data-driven implementations of these proposed IRL algorithms are presented. Consequently, this research contributes to identifying unknown reward functions and optimal controls by conducting demonstrations. Our analysis shows that the stability and convergence of MASs are thoroughly ensured. The effectiveness of the given algorithms is demonstrated via simulation results.