SMFPS: A semi-supervised multi-modal fusion method for RGBD particle segmentation of industrial materials

Abstract

In industrial production, the particle size distribution of materials is critical for adjusting production parameters and maintaining quality control, directly affecting manufacturing efficiency and product quality. However, RGB-based segmentation methods often face difficulties in accurately segmenting particles due to complex surface textures and uneven lighting. In industrial scenarios where annotated data are scarce, the accuracy and generalization capability of traditional segmentation methods are further constrained. To address this, we propose a Semi-supervised Multi-modal Fusion Particle Segmentation (SMFPS) framework that achieves high-precision segmentation with low annotation cost. We introduce a Feature Calibration Adaptive Fusion Module (FCAFM) to perform cross-modal fusion through spatial similarity correction and channel attention. In addition, we design a multi-modal semi-supervised data augmentation approach, CoverMix, which generates occlusion perturbations using depth information to enhance semi-supervised learning. Experiments on a constructed industrial material dataset demonstrate that SMFPS achieves a mean intersection over union (mIoU) of 84.44% using only 20% labeled data, matching or exceeding fully supervised methods. This provides an efficient, accurate, and low-cost solution for on-line particle size detection in industrial materials.