FITrans: Skin Lesion Segmentation Based on Feature Integration and Transformer

Abstract

Automatic segmentation of skin lesions in dermatoscopic images is of great significance for clinical diagnosis and evaluation of melanoma. Most existing skin lesion segmentation methods overfit the primary visual region in skin lesion images but ignore the edge details. When the boundary of the lesion is blurred and the contrast between the lesion and the background is low, it is difficult to obtain an accurate segmentation. To solve this problem, this paper proposes a network model FITrans that combines multi-level feature aggregation and Transformer. Firstly, the convolution neural network uses additional classification information and boundary information to extract discriminative context and edge features by weakly supervised weight feedback and explicit edge guidance, respectively. Secondly, the model integrates the global-level context features, low-level edge features, and high-level semantic features extracted from the backbone network. The integrated multi-level features can compensate for the shortcomings of single-feature segmentation and enhance the segmentation effect of edge areas. On this basis, the Transformer model codes the serialized integration features, which not only make up for the problem that the Transformer does not make full use of the low-level features, but also extracts rich information on distance dependence between pixels. Combining the multi-level integration features with the Transformer can further enhance the segmentation effect of edge regions. In this paper, FITrans are used to analyze the effect of different levels of features on segmentation, and to verify the validity of multi-level feature integration and distance-dependent information. FITrans achieved 88.6 and 81.7 Jaccard coefficients (JAs) on ISIC2016 and ISIC2017, which exceed the five mainstream skin lesion segmentation methods. The experimental results show that the FITrans model can effectively improve the accuracy of target edge segmentation, and is more helpful for the accurate segmentation of complex skin damage images.