Semantic segmentation of hyperspectral images with the fusion of LiDAR data

Semantic segmentation is an emerging field in the computer vision community where one can segment and label an object all at once. In this paper, we propose a semantic segmentation algorithm that takes into account both the hyperspectral images and the LiDAR data. In our segmentation framework, we propose a new energy function that is composed of two terms: a unary energy term and a pairwise energy term. The unary energy term provides the segmentation maps for the hyperspectral data as well as for the LiDAR data which is explained with Fisher Vectors. The pairwise spatial term uses both the UTM coordinates as well as the LiDAR data. Finally, the system is solved with graph-cuts. We report the effect of the parameters in energy minimization and show that the best results are achieved with an SVM-MRF classifier among the several classifiers.