Coarse-to-Fine High-Order Network for Hyperspectral and LiDAR Classification

Abstract

The fusion of hyperspectral and light detection and ranging (LiDAR) data could significantly improve land-cover classification performance. Most existing feature fusion methods focus on “late fusion” or “halfway feature interaction fusion” methods, which treat LiDAR and hyperspectral data as model inputs while overlooking the redundancy between hyperspectral imagery (HSI) and LiDAR features. In addition, the unique characteristics of HSI and LiDAR data, combined with the complexity of land-cover backgrounds, make it challenging to accurately describe their properties. High-order deep features, as a deep statistical representation, could effectively capture the statistical characteristics of these land covers. Based on this, this article focuses on the unique characteristics of HSI and LiDAR data, as well as the distinguishability of features, and designs a progressive hyperspectral and LiDAR collaborative classification method, named coarse-to-fine high-order network (CHNet). In the coarse stage, HSI data redundancy reduction and LiDAR feature reconstruction are performed in either the frequency domain or the spatial domain to ensure the effectiveness of subsequent feature fusion. The fine stage focuses primarily on selective feature fusion and discriminability enhancement, introducing gating mechanisms, deep expert systems, and high-order feature statistics. This approach enhances the discriminability of the fused features while simultaneously reducing their dimensionality. The proposed method, based on a “redundancy removal-feature learning” mechanism, captures more effective deep features by accounting for the different imaging mechanisms of multisource data and the complex background of land covers, ultimately improving the effectiveness of land-cover classification. Experimental results on three public datasets and one self-constructed dataset demonstrate that CHNet achieves superior performance. The code is available at https://github.com/RSIP-NJUPT/CHNet.