Maritime scene matching for inter-robot localization across surface and underwater domains

Autonomous underwater vehicles (AUVs) play a crucial role across various sectors, including oil and gas production, civil engineering, and defense. However, underwater localization remains a significant challenge, limiting the widespread adoption of AUVs in these fields. A common strategy to address this challenge is to deploy a fleet of Uncrewed Surface Vessels (USVs), which are easier to localize on the surface, alongside AUVs to help anchor their position estimates. These methods typically rely on acoustic pinging among the robots to relay position-related data. Unfortunately, acoustic pinging requires synchronized clocks and a clear line of sight, making it difficult to deploy large-scale, decentralized teams, particularly in littoral (i.e. near-shore) environments.

To bridge this gap, we propose an alternative approach that is resolvable over asynchronous, intermittent communications. By leveraging the fact that many human-made structures in littoral environments are visible both above and below the waterline, our method automatically detects correspondences between above-water LiDAR scenes and underwater sonar scenes. First, we convert underwater and above-water data into a common representation, generate descriptors to find commonalities, and then apply point cloud registration tools to find rigid body transformations between them. Lastly, we apply pairwise consistent measurement set maximization (PCM) as a robust outlier rejection system. Our results demonstrate that our solution to this novel Maritime Scene Matching (MSM) problem is both robust to outliers and effective in localizing sonar scenes with an accuracy of less than two meters. Datasets are collected using a single robot equipped with underwater imaging sonar and above-water LiDAR. We have made our real-world datasets, hardware designs, and open-source code available to promote reproducibility and to encourage broader community engagement with the MSM problem. Opensource code: https://github.com/jake3991/maritime-scene-matching.