Label-guided graph learning network via two-stage cross-modal fusion for multi-label skin disease diagnosis

The accurate diagnosis of skin diseases relies on combining cortical lesion morphological features from clinical and dermoscopic data with deep subcutaneous lesion characteristics from ultrasound data. However, current multi-modal diagnostic methods mainly emphasize clinical and dermoscopic data analysis, lacking a thorough exploration of subcutaneous tissue features provided by skin ultrasound. Additionally, existing research often overlooks the analysis of relationships between skin labels and categories across different diagnostic tasks, constraining model performance as the number of target tasks grows. This paper proposes a label-guided graph learning network (LGL_Net) based on two-stage cross-modal fusion to achieve accurate diagnosis of skin diseases. Specifically, this paper first establishes a two-stage cross-modal fusion unit (TC_Fusion) to enable the fusion and transmission of morphological features from clinical data and deep lesion features from ultrasound data. Subsequently, a label-guided graph learning unit (LGL_Unit) is constructed to explore the correlations between multi-label data of skin lesions by building a graph convolutional network (GCN) at the label and category levels, thereby improving the accuracy of various skin disease diagnostic tasks. Extensive experiments were conducted on a public dataset (including clinical and dermoscopic data) and a private dataset (including clinical and ultrasound data). The experimental results demonstrate that the proposed method achieves optimal performance on tasks like pathological diagnosis (PG), benign or malignant diagnosis (BM), and 7-Point Checklist scoring (7 PC), offering a new approach for skin disease diagnosis. Our code is available at: https://github.com/Zhaocheng1/LGL-Net.