Twin deformable point convolutions for airborne laser scanning point cloud classification

Abstract

Thanks to the application of deep learning technology in point cloud processing of the remote sensing field, point cloud classification has become a research hotspot in recent years. Although existing solutions have made unprecedented progress, they ignore the inherent characteristics of point clouds in remote sensing fields that are strictly arranged according to latitude, longitude, and altitude, which brings great convenience to the segmentation of point clouds in remote sensing fields. To consider this property cleverly, we propose novel convolution operators, termed Twin Deformable point Convolutions (TDConvs), which aim to achieve adaptive feature learning by learning deformable sampling points in the latitude–longitude plane and altitude direction, respectively. First, to model the characteristics of the latitude–longitude plane, we propose a Cylinder-wise Deformable point Convolution (CyDConv) operator, which generates a two-dimensional cylinder map by constructing a cylinder-like grid in the latitude–longitude direction, and then performs adaptive feature sampling on the cylinder map by deformable offset learning. Furthermore, to better integrate the features of the latitude–longitude plane and the spatial geometric features, we perform a multi-scale fusion of the extracted latitude–longitude features and spatial geometric features, and realize it through the aggregation of adjacent point features of different scales. In addition, a Sphere-wise Deformable point Convolution (SpDConv) operator is introduced to adaptively offset the sampling points in three-dimensional space by constructing a sphere grid structure, aiming at modeling the characteristics in the altitude direction. Experiments on existing popular benchmarks conclude that our TDConvs achieve the best segmentation performance, surpassing existing advanced methods such as RFFS-Net and MCFN. Specifically, TDConvs achieves 73.4% mF1 on the ISPRS Vaihingen 3D dataset, which is 4.8% higher than the baseline. Details of the datasets used and the code is available on https://github.com/WingkeungM/TDConvs.