UTMT: Semi-supervised segmentation of surgical endoscopic images based on Uncertainty guided Twin Mean Teacher

Semantic segmentation of surgical endoscopic images plays an important role in surgical skill analysis and guidance. Deep learning based on fully supervised learning has achieved remarkable performance in this task, but it relies on a large amount of training images with pixel-level annotations that are time-consuming and difficult to collect. To reduce the annotation cost, we propose a novel semi-supervised segmentation framework based on Uncertainty guided Twin Mean Teacher (UTMT) for surgical endoscopic image segmentation. UTMT has two parallel teacher–student structures to deal with unannotated training images, where each student is supervised by pseudo-labels obtained by not only its mean teacher, but also its fellow student. The combination of mean teacher and fellow student can reduce the inherent bias of a single model and improve the quality of pseudo-labels. In addition, considering that the pseudo-label may be noisy, we propose an uncertainty-based correction method to emphasize high-confidence pseudo-labels obtained from different networks and suppress the unreliable parts for more robust learning. Experimental results on two public surgical endoscopic image datasets demonstrated that UTMT significantly improved the segmentation performance when only 1% or 5% of the training images were labeled, and it outperformed six state-of-the-art semi-supervised segmentation methods. Furthermore, compared with fully supervised learning, UTMT achieved a similar performance while reducing the annotation cost by 90% on the two datasets.