Dense image-mask attention-guided transformer network for jaw lesions classification and segmentation in dental cone-beam computed tomography images

Abstract

Automatic segmentation and classification of jaw lesions from cone-beam computed tomography (CBCT) images are crucial in computer-assisted diagnosis and treatment planning for oral and maxillofacial (OMF) surgery. However, the evolutionary nature of jaw lesions and their morphological diversity pose significant challenges to both segmentation and classification tasks. Although existing deep learning-based works have achieved promising results on segmentation and classification of other types of lesions, they often consider the two tasks separately, thereby overlooking the strong guidance that lesion masks can provide in determining lesion categories. In this manuscript, we propose a dense image-mask attention-guided transformer network for end-to-end jaw lesions classification and segmentation in 3D CBCT images based on a multi-task learning (MTL) architecture. Specifically, we design multi-dimension attention (MDA) and multi-scale attention (MSA) modules to incorporate dense features from different dimensions and scales, explicitly enhancing the guidance of lesion segmentation for classification decisions. Furthermore, to effectively encode long-term contextual information, we employ a transformer as the classification decoder and design a 3D positional embedding method to preserve the 3D positional information of sequential feature inputs for the transformer. Finally, we design a task merge module that employs a per-lesion inference strategy to assign a category to each lesion instance. A large in-house dataset consisting of 358 CBCT scans with five types of jaw lesions is constructed to evaluate the proposed method. The experimental results show a binary segmentation DICE score of 90%, a mean classification accuracy of 89.23%, and a multi-class segmentation DICE score of 79.06%, surpassing many state-of-the-art methods.