Learning Visual Navigation System in Simulation for Autonomous Ground Vehicles in Real World

Navigation for autonomous ground vehicles (AGV) should be accurate and quick. Traditional navigation systems, consisting of perception, planning, and control, are unable to use noisy visual images efficiently on a power-limited computation unit. These systems also require lots of parameter-tuning work when deployed on a new robot. By contrast, end-to-end approaches, that directly map sensor information and robot state to planned trajectories, have the potential to navigate autonomous ground vehicles on edge computation devices and possess far fewer manually-tuned parameters. However, collecting data on real robots and labeling the data for training is time-consuming and costly. Therefore, many approaches turn to automatic data labeling and collection in the simulation environment. Motivated by a learning-based navigation system for drones, we present a sim-to-real learning-based navigation pipeline for AGVs where the model is solely trained in simulation environments (Gazebo and UE4) and directly deployed to a real AGV. Results show that after training, the system achieves a high success rate in both simulation and real-world cases, indicating the great potential of this learning pipeline.