Multi-frequency shared-feature-learning based diffusion model for removing surgical smoke

Surgical smoke in laparoscopic surgery can deteriorate visibility for surgeons. This work aims to simultaneously remove the surgical smoke and restore true-to-life image colors with deep learning. However, deep learning-based smoke removal remains a challenge due to: 1) the non-homogeneous distribution of surgical smoke, 2) higher frequency modes being hindered from being learned due to spectral bias. In this work, we propose the multi-frequency shared-feature-learning based conditional diffusion model with adaptive smoke attention for removing surgical smoke. The proposed model learns to map both the smoky and smokeless images into a shared inherent feature by the forward learning and synthesize the smokeless image by the reverse learning, and the input noisy image used for the forward learning is wrapped by the smoke attention learning to ease sampling steps and facilitate shared feature optimization. The smoke attention learning employs smoke segmentation and convolutional block attention modules to capture the non-homogeneous features of smoke. The multi-frequency learning is introduced to incorporate with shared feature learning to enhance the mid-to-high frequency features. In addition, the multi-task learning incorporates shared feature loss, smoke perception loss, dark channel prior loss, and contrast enhancement loss to help the model optimization. The experimental results show that the proposed method outperforms other state-of-the-art methods on both synthetic/real laparoscopic surgical images, with the potential to be embedded in laparoscopic devices for de-smoking.