AttenScribble: Attention-enhanced scribble supervision for medical image segmentation

Abstract

The success of deep networks in medical image segmentation relies heavily on massive labeled training data. However, acquiring dense annotations is a time-consuming process. Weakly supervised methods normally employ less expensive forms of supervision, among which scribbles started to gain popularity lately thanks to their flexibility. However, due to the lack of shape and boundary information, it is extremely challenging to train a deep network on scribbles that generalize on unlabeled pixels. In this paper, we present a straightforward yet effective scribble-supervised learning framework. Inspired by recent advances in transformer-based segmentation, we create a pluggable spatial self-attention module that could be attached on top of any internal feature layers of arbitrary fully convolutional network (FCN) backbone. The module infuses global interaction while keeping the efficiency of convolutions. Descended from this module, we construct a similarity metric based on normalized and symmetrized attention. This attentive similarity leads to a novel regularization loss that imposes consistency between segmentation prediction and visual affinity. This attentive similarity loss optimizes the alignment of FCN encoders, attention mapping and model prediction. Ultimately, the proposed FCN+Attention architecture can be trained end-to-end guided by a combination of three learning objectives: partial segmentation loss, customized masked conditional random fields, and the proposed attentive similarity loss. Extensive experiments on public datasets (ACDC and CHAOS) showed that our framework not only outperforms existing state-of-the-art but also delivers close performance to fully-supervised benchmarks. The code is available at https://github.com/YangQinzhu/AttenScribble.git.