Unsupervised multimodal feature learning for semantic image segmentation

In this paper, we address the semantic segmentation problem using single-layer networks. This network is used for unsupervised feature learning for the available RGB image and the depth image. A significant contribution of the proposed approach is that the dictionary is selected from the existing samples using the L2, 1 optimization. Such a dictionary can capture more meaningful representative samples and exploit intrinsic correlation between features from different modalities. The experimental results on the public NYU dataset show that this strategy dramatically improves the classification performance, compared with existing dictionary learning approach. In addition, we perform experimental verification using the practical robot platforms and show promising results.