Sample-efficient multi-objective reinforcement learning for hybrid impulsive information consensus in multi-agent systems

Abstract

A hybrid impulsive-control approach integrating multi-objective reinforcement learning is proposed in this work to tackle the information consensus regulation problem in multi-agent systems. The proposed method is developed under a hybrid impulsive control framework, which explicitly considers both time-varying communication delays and impulsive delays inherent in the system, and incorporates stabilizing impulses to mitigate instability caused by continuous dynamics. The impulsive-control sequence is dynamically adjusted by a multi-objective reinforcement learning (MORL) agent, thereby achieving balanced optimization among multiple objectives—including communication cost and state deviation. Sample efficiency is enhanced through a manually designed prior control policy, by which the agent is endowed with decision-making capabilities via early-stage imitation learning, while a dual experience-replay buffer is configured to ensure seamless transition from imitation-guided learning to autonomous exploration; these mechanisms constitute the Prior Knowledge-Guided Multi-Objective Reinforcement Learning (PKG-MORL) algorithm proposed herein. Experience utilization is improved through the Adaptive Informative Prioritized Experience Replay (AIPER) mechanism, where temporal-difference error, information gain, and sampling frequency are jointly incorporated to refine the sampling strategy, thereby accelerating policy convergence and enhancing training efficiency. Experimental evaluations verify that the proposed hybrid impulsive-control method effectively achieves information consensus control under various preference scenarios and yields higher-quality approximated Pareto fronts than prevailing MORL algorithms.