Multi-scale CNN-Swin transformer network with boundary supervision for multiclass biomarker segmentation in retinal OCT images

Retinal biomarker morphology is closely associated with a variety of chronic ophthalmic diseases, in which biomarker localization and segmentation in optical coherence tomography (OCT) play a key role in the diagnosis of retina-related diseases. Although great progress has been made in deep learning based OCT biomarker segmentation, several challenges still exist. Due to issues such as image noise or class imbalance, retinal biomarkers affect the model’s recognition of other biomarkers. Moreover, small biomarkers are prone to lose accuracy during downsampling. And most existing methods rely on convolutional neural networks, which make it challenging to obtain the global context due to locality of convolution. Benefiting from the Swin Transformer with powerful modeling capabilities, we propose MSCS-Net (Multi-scale CNN-Swin Network), a network for OCT biomarker segmentation, which effectively combines CNN and Swin Transformer and integrates them in parallel into a dual-encoder structure. Specifically, an edge detection path is added alongside to enhance the localization of biomarkers at the edges. For the Swin Transformer branch, considering the irregular distribution of most OCT biomarkers, a new windowing partition is performed in the Swin Transformer to capture the features more efficiently. Meanwhile, we design a Feature Dimensionality Reduction Module to extensively collect the information of small-scale biomarkers. To effectively integrate information from two scales, we design a Transformer Cross Fusion Module to finely fuse the global and local feature information from the two-branch encoders. We validate the proposed approach on local and public datasets, and the experimental results demonstrate the effectiveness of the proposed framework.