Semantic Exploration and Dense Mapping of Complex Environments Using Ground Robot With Panoramic LiDAR-Camera Fusion

Abstract

This paper presents a system for autonomous semantic exploration and dense semantic target mapping of a complex unknown environment using a ground robot equipped with a panoramic LiDAR-camera system. Existing approaches often struggle to strike a balance between collecting enough high-quality observations from multiple view angles and avoiding unnecessary repetitive traversal. To fill this gap, we propose a complete system that combines mapping and planning. We first redefine the task as completing both geometric coverage and semantic viewpoint observation. Subsequently, we manage semantic and geometric viewpoints separately and propose a novel Priority-driven Decoupled Local Sampler to generate local viewpoint sets. This allows for explicit multi-view semantic inspection and voxel coverage without unnecessary repetition. Building on this, we develop a hierarchical planner that ensures efficient global coverage. In addition, we propose a Safe Aggressive Exploration State Machine, which allows the robot to extend its exploration path planning into unknown areas while ensuring the robot's safety with recovery behavior and adaptive sampling. Our system includes a modular semantic target mapping component designed to utilize odometry and point clouds from existing SLAM algorithms, enabling point-cloud-level dense semantic target mapping. We validate our approach through extensive experiments in both realistic simulations and complex real-world environments. Simulation results demonstrate that our planner achieves faster exploration and shorter travel distances while guaranteeing a specified number of multi-view inspections. Real-world experiments further confirm the effectiveness of the system in achieving accurate dense semantic object mapping of unstructured environments.