Diagnosing Skin Lesion Using Multi-Modal Analysis

The vast expansion of modalities in the medical field has helped identify and cure many diseases. Each modality has its specific characteristic feature. Multi-modal data analysis fuses different characteristic features and generates helpful information. With recent advances, deep learning has gain much importance in analyzing medical images. Generally, in the medical field, images are used for extracting features. Multi-modal image analysis uses deep learning models. The knowledge of the patient history and laboratory data helps create a simple clinical context. Combining information about the patient history and distinct modalities can be effective in obtaining an accurate diagnosis. The proposed model performs reasonably better than the other deep learning models, taking the image as input and additional helpful information about the patient. We have used the PAD-UFES-20 dataset for this study.