An Integrated Multimodal Deep Learning Framework for Accurate Skin Disease Classification

S. Hamida, Driss Lamrani, Mohammed Amine Bouqentar, Oussama El Gannour, B. Cherradi
{"title":"An Integrated Multimodal Deep Learning Framework for Accurate Skin Disease Classification","authors":"S. Hamida, Driss Lamrani, Mohammed Amine Bouqentar, Oussama El Gannour, B. Cherradi","doi":"10.3991/ijoe.v20i02.43795","DOIUrl":null,"url":null,"abstract":"In order to effectively treat skin diseases, an accurate and prompt diagnosis is required. In this article, a novel method for classifying skin disorders using a multimodal classifier is presented. The proposed classifier utilizes multiple information sources to enhance the accuracy of disease classification. It incorporates images of skin lesions and patient-specific data. The multimodal classifier simultaneously classifies diseases by combining image and structured data inputs. The effectiveness of the proposed classifier was evaluated using the ISIC 2018 dataset, which includes images and clinical data for seven categories of skin diseases. The results indicate that the proposed model outperforms conventional single-modal and single-task classifiers, achieving an accuracy of 98.66% for image classification and 94.40% for clinical data classification. In addition, we compare the performance of the proposed model with that of other methodologies, demonstrating its superiority. Despite yielding promising results, the proposed method has limitations in terms of data requirements and generalizability. Future research directions include incorporating additional information sources, investigating genetic data integration, and applying the method to various medical conditions. This study illustrates the potential of integrating multimodal techniques with transfer learning in deep neural networks to enhance the classification accuracy of cutaneous diseases.","PeriodicalId":507997,"journal":{"name":"International Journal of Online and Biomedical Engineering (iJOE)","volume":"606 ","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Online and Biomedical Engineering (iJOE)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3991/ijoe.v20i02.43795","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

In order to effectively treat skin diseases, an accurate and prompt diagnosis is required. In this article, a novel method for classifying skin disorders using a multimodal classifier is presented. The proposed classifier utilizes multiple information sources to enhance the accuracy of disease classification. It incorporates images of skin lesions and patient-specific data. The multimodal classifier simultaneously classifies diseases by combining image and structured data inputs. The effectiveness of the proposed classifier was evaluated using the ISIC 2018 dataset, which includes images and clinical data for seven categories of skin diseases. The results indicate that the proposed model outperforms conventional single-modal and single-task classifiers, achieving an accuracy of 98.66% for image classification and 94.40% for clinical data classification. In addition, we compare the performance of the proposed model with that of other methodologies, demonstrating its superiority. Despite yielding promising results, the proposed method has limitations in terms of data requirements and generalizability. Future research directions include incorporating additional information sources, investigating genetic data integration, and applying the method to various medical conditions. This study illustrates the potential of integrating multimodal techniques with transfer learning in deep neural networks to enhance the classification accuracy of cutaneous diseases.
用于准确皮肤病分类的多模态深度学习综合框架
为了有效治疗皮肤病,需要准确及时的诊断。本文介绍了一种利用多模态分类器对皮肤病进行分类的新方法。所提出的分类器利用多种信息源来提高疾病分类的准确性。它结合了皮肤病变的图像和病人的具体数据。多模态分类器通过结合图像和结构化数据输入,同时对疾病进行分类。使用 ISIC 2018 数据集对所提议的分类器的有效性进行了评估,该数据集包括七类皮肤病的图像和临床数据。结果表明,所提出的模型优于传统的单模态和单任务分类器,其图像分类准确率达到 98.66%,临床数据分类准确率达到 94.40%。此外,我们还将所提模型的性能与其他方法进行了比较,证明了其优越性。尽管提出的方法取得了可喜的成果,但在数据要求和通用性方面仍有局限。未来的研究方向包括纳入更多信息源、研究基因数据整合以及将该方法应用于各种医疗状况。本研究说明了在深度神经网络中将多模态技术与迁移学习相结合以提高皮肤疾病分类准确性的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信