Skill-abstracting continual reinforcement learning for safe, efficient, and comfortable autonomous driving through vehicle–cloud collaboration

Safe, efficient, and comfortable autonomous driving is essential for high-quality transport service in an open road environment. However, most existing driving strategy learning approaches for autonomous driving struggle with varying driving environments, only working properly under certain scenarios. Therefore, this study proposes a novel hierarchical continual reinforcement learning (RL) framework to abstract various driving patterns as skills and support driving strategy adaptation based on vehicle-cloud collaboration. The proposed framework leverages skill abstracting in the cloud to learn driving skills from massive demonstrations and store them as deep RL models, mitigating catastrophic forgetting and data imbalance for driving strategy adaptation. Connected autonomous vehicles’ (CAVs) driving strategies are sent to the cloud and continually updated by integrating abstracted driving skills and interactions with parallel environments in the cloud. Then, CAVs receive updated driving strategies from the cloud to interact with the real-time environment. In the experiment, high-fidelity and stochastic environments are created using real-world pavement and traffic data. Experimental results showcase the proposed hierarchical continual RL framework exhibits a 34.04% reduction in potentially hazardous events and a 9.04% improvement in vertical comfort, compared to a classical RL baseline, demonstrating superior driving performance and strong generalization capabilities in varying driving environments. Overall, the proposed framework reinvigorates streaming driving data, prevailing motion planning models, and cloud computation resources for life-long driving strategy learning.