3D semantic segmentation: Cluster-based sampling and proximity hashing for novel class discovery

Abstract

Novel Class Discovery (NCD) in 3D semantic segmentation is crucial for applications requiring the ability to learn and segment previously unknown classes in point cloud data, such as autonomous driving and urban planning. Traditional 3D semantic segmentation methods often build upon a fixed set of known classes, which restricts their ability to discover classes not covered in the original training data. To overcome these limitations, we propose a novel framework specifically designed for NCD in 3D semantic segmentation. The framework integrates the Voxel-Geometry Data Integration module, the Cluster-based Representative Sampling module, the Neighborhood Spatial Partitioning module, and the Spatial Feature Attention Mechanism. These modules collectively enhance the model’s capability to integrate spatial and geometric information, identify key representative points, map neighborhoods effectively, and synthesize localized and global features. Experimental results on benchmark datasets, including S3DIS, Toronto-3D, SemanticSTF, and SemanticPOSS, demonstrate the proposed method’s superior performance in discovering novel classes and improving overall segmentation quality. For instance, in the SemanticPOSS-$4^0$ split, the method achieves a mean Intersection over Union (mIoU) of 43.68% for novel classes, compared to 35.70% achieved by NOPS. These results highlight the framework’s effectiveness in handling complex scenes and its potential to advance NCD in 3D semantic segmentation.