Point-based Attention Convolutional Neural Networks for Point Clouds Semantic Segmentation

Abstract

Convolutional neural network (CNNs) have achieved success in processing data with regular grid structures, demonstrating the great potential of applying CNN to point cloud data. However, the disorder and irregularity of 3D point cloud data hinder this progress. To address this issue, we propose a point-based attention convolutional neural network, which consists of a dynamic attention convolution module (DAC) and a point-based feature relation matrix aggregation module (PRA). DAC is used to extract features. At each layer of the network, DAC recomputes the dynamic update graph and assigns attention weights to each edge across the feature space of different points, and finally updates the features by the weighted sum of adjacent points. PRA utilizes the raw and aggregated features of points to generate a global relation matrix, which can adjust the aggregated features for biases from DAC, while obtaining long-range contextual information. Our network structure consists of an encoder and a decoder, and in order to enhance the results of multi-scale feature fusion, we optimize the feature fusion process after upsampling to form a more detailed end-to-end trainable network. Through segmentation and classification experiments on challenging 3D point cloud benchmarks, we demonstrate that our algorithm can meet or outperform the performance of existing state-of-the-art methods.