Long short term robot safe control framework based on hidden action space heuristic soft actor–critic

Real-time safety control for robots in dynamic environments is a critical and challenging problem in robotics. With the advent of intelligent manufacturing, the demand for advanced safety control technologies in robotics has steadily increased. Robot planning typically involves global and local approaches, but both face limitations in real-time safety control in dynamic environments with unknown obstacles. Recent hybrid frameworks have shown progress, but challenges persist, including limited perception capabilities and poor coordination between global and local components. To address these challenges, this work proposes a novel long short term safety control framework leveraging reinforcement learning for decision-making. Perception and planning are decoupled into long-term and short-term components, with long-term perception utilizing unsupervised clustering DBSCAN for structured environment information and short-term perception enhancing efficiency through prior knowledge. Long-term planning provides reference trajectories based on static environments, while short-term planning adjusts these trajectories in real time for local safety using control barrier functions. Based on hidden action heuristic soft actor–critic and curriculum learning, the decision-making mechanism ensures safety during obstacles or attacks and maximizes robot efficiency without compromising safety. Experiments are conducted with 10,000 randomized obstacle collision scenarios, and our framework is compared with four methods, including SAC and manually designed trajectory adjustment. The results demonstrate that our approach outperforms these methods in both safety performance and operational efficiency. Finally, the system is successfully implemented in a physical environment, showcasing its practical potential for real-world applications.