Semantic Visual Segmentation of a Mobile Robot Environment Using Deep Learning Model

Abstract

This paper addresses the use of deep learning techniques in 3D point cloud labeling of environment representations for the task of a semantic visual localization of mobile robots. In contrast to standard problems resolved with Convolutional Neural Networks (CNNs), the paper deals with applying CNNs to segment point clouds that are, unlike images, unordered and unstructured. The used point clouds contain laser measurements of 3D positions (x,y,z) as well as captured RGB camera images from the scanned scene to colorize the point cloud (RGB values). The main focus of the paper is on implementation and evaluation of a hand-crafted convolution layer and the ConvPoint CNN architecture that introduces continuous convolutions for point cloud processing. The solution was implemented in the Python programming language using the PyTorch deep learning framework.