Refining the granularity of smoke representation: SAM-powered density-aware progressive smoke segmentation framework

Accurately segmenting smoke remains a challenging task due to its non-rigid morphology and semi-transparent nature, which causes pixel blending between smoke and background, leading to intertwined representations. Among these issues, smoke density plays a crucial role for refine the granularity of smoke representation, yet it has been largely overlooked in previous research. In this work, we introduce the SAM-powered (Segment Anything Model [1]) DenSity-Aware Progressive Smoke Segmentation method (DenSiSeg). For smoke regions, we construct a background feature prototype to refine smoke mask labels into fine-grained density information using a per-pixel metric. Following this, soft-contrastive learning and progressive evolving strategies are devised to smoothly and iteratively refine the feature distribution of smoke at different density levels. For background regions, knowledge transfer based on the vision foundation model is employed, harnessing the world knowledge within the foundation model to enhance the understanding of diverse background. Extensive experiments on several public datasets demonstrate that the proposed DenSiSeg method significantly outperforms state-of-the-art methods. The code will be available on https://github.com/Caoyichao/DenSiSeg.