DAMAF: dual attention network with multi-level adaptive complementary fusion for medical image segmentation

Abstract

Transformers have been widely applied in medical image segmentation due to their ability to establish excellent long-distance dependency through self-attention. However, relying solely on self-attention makes it difficult to effectively extract rich spatial and channel information from adjacent levels. To address this issue, we propose a novel dual attention model based on a multi-level adaptive complementary fusion mechanism, namely DAMAF. We first employ efficient attention and transpose attention to synchronously capture robust spatial and channel cures in a lightweight manner. Then, we design a multi-level fusion attention block to expand the complementarity of features at each level and enrich the contextual information, thereby gradually enhancing the correlation between high-level and low-level features. In addition, we develop a multi-level skip attention block to strengthen the adjacent-level information of the model through mutual fusion, which improves the feature expression ability of the model. Extensive experiments on the Synapse, ACDC, and ISIC-2018 datasets demonstrate that the proposed DAMAF achieves significantly superior results compared to competitors. Our code is publicly available at https://github.com/PanYging/DAMAF.