Receding-Horizon Reinforcement Learning for Time-Delayed Human–Machine Shared Control of Intelligent Vehicles

Abstract

Human–machine shared control has recently been regarded as a promising paradigm to improve safety and performance in complex driving scenarios. One crucial task in shared control is dynamically optimizing the driving weights between the driver and the intelligent vehicle to adapt to dynamic driving scenarios. However, designing an optimal human–machine shared controller with guaranteed performance and stability is challenging due to nonnegligible time delays caused by communication protocols and uncertainties in driver behavior. This article proposes a novel receding-horizon reinforcement learning approach for time-delayed human–machine shared control of intelligent vehicles. First, we build a multikernel-based data-driven model of vehicle dynamics and driving behavior, considering time delays and uncertainties of drivers' actions. Second, a model-based receding horizon actor–critic learning algorithm is presented to learn an explicit policy for time-delayed human–machine shared control online. Unlike classic reinforcement learning, policy learning of the proposed approach is performed according to a receding-horizon strategy to enhance learning efficiency and adaptability. In theory, the closed-loop stability under time delays is analyzed. Hardware-in-the-loop experiments on the time-delayed human–machine shared control of intelligent vehicles have been conducted in variable curvature road scenarios. The results demonstrate that our approach has significant improvements in driving performance and driver workload compared with pure manual driving and previous shared control methods.