Study on multi-modal sensor system based sematic navigation map building

Localization technology is essential for robots. The map created to recognize the location mainly contains metric information. However, in a changing environment, a Semantic Map containing Semantic object information is required a multi-modal sensor composed of multiple types and multiple sensors[RGBD, thermal, night vision, global shutter camera, microphone, 16 channel laser sensor(=Lidar)] was created for semantic information recognition and semantic map creation in various environments, and calibration was performed to integrate the coordinate system. After that, we introduce the method of generating the metric map according to the configuration of the multi-modal sensor. Also, we propose a method to obtain a single accurate location by integrating the location recognition results obtained from various maps. This can be used to specify the position of the semantic object. Finally, it can be expected that the semantic object and the semantic map information obtained through the multi-modal sensor can be used for various different sensor configurations and various types of robots.