Minute-scale training for microrobot navigation

Abstract

Microrobots hold considerable potential for various applications in which targeted navigation is a basic requirement. Deep reinforcement learning has recently emerged as a powerful paradigm for fully autonomous microrobot navigation. Yet, current deep reinforcement learning-based approaches pay limited attention to learning efficiency and effectiveness, requiring hours to days for model training. Consequently, this impedes both rapid practical deployment and parameter optimization. To address these challenges, we present a learning framework that enables effective microrobot navigation policies to be trained within minutes. Here in the proposed framework, we develop a fully vectorized simulator with more than 10,000 artificial vascular environments, parallelizing dynamics, ray-casting-based visual feature extraction and feasibility checks across thousands of environments to achieve roughly 190,000 transitions per second. To achieve effectiveness in the fast training, we propose a task-shaping-regularization reward framework. The task-shaping-regularization framework accelerates convergence, improves final performance, reduces action variation by at least 33.7% and increases obstacle clearance by at least 2.1% across all evaluated scenarios. Results show that the proposed learning framework reduces training time to under 10 minutes, while supporting zero-shot deployment across distinct microrobot types and navigation scenarios. Collectively, this framework can substantially shorten the design loop and accelerate the deployment of autonomous microrobots. A vectorized simulator and structured reward framework enable microrobot navigation policies to be trained within minutes and transferred without retraining across robots and environments.