FCAformer: Fuzzy-Enhanced Class-Aware Attention Based Transformer for Weakly Supervised Histopathology Image Segmentation

Abstract

Pixel-level histopathology image segmentation plays a vital role in computational pathology, and weakly supervised segmentation methods, which rely solely on image-level labels, have shown great potential. However, most existing weakly supervised segmentation methods are limited by the fixed receptive field of convolutional neural networks, and overlook the uncertainty of the distribution of different tissue types and the fuzziness of class boundaries, resulting in limited segmentation effects. To address these problems, we propose a fuzzy-enhanced class-aware attention based Transformer (FCAformer) for weakly supervised histopathology image segmentation. FCAformer employs the Transformer architecture for model global contextual information, which effectively alleviates the limitation of fixed receptive field on the size of attention map in traditional methods. Subsequently, FCAformer integrates fuzzy system to model the uncertainty in histopathology images. Specifically, it assigns membership function to the output feature map of the last layer of Transformer encoder, generates fuzzy membership matrix, extracts fuzzy features by combining three fuzzy rules, and finally fuses these features to generate fuzzy attention map. This attention map guides the network to learn the characteristics of different tissue types and improve the fuzziness of class boundaries, thereby improving the modeling ability of the model on uncertain tissue distribution and fuzzy areas. In addition, based on the idea of contrastive learning, we design contrastive class token (CCT) loss to further enhance the distinguishability between different class labels. Extensive experiments on LUAD-HistoSeg and BCSS-WSSS datasets demonstrate that FCAformer achieves state-of-the-art segmentation performance in weakly supervised segmentation tasks.