Analysis of nailfold capillaroscopy images with artificial intelligence: Data from literature and performance of machine learning and deep learning from images acquired in the SCLEROCAP study
Lutfi Ozturk , Charlotte Laclau , Carine Boulon , Marion Mangin , Etheve Braz-ma , Joel Constans , Loubna Dari , Claire Le Hello
{"title":"Analysis of nailfold capillaroscopy images with artificial intelligence: Data from literature and performance of machine learning and deep learning from images acquired in the SCLEROCAP study","authors":"Lutfi Ozturk , Charlotte Laclau , Carine Boulon , Marion Mangin , Etheve Braz-ma , Joel Constans , Loubna Dari , Claire Le Hello","doi":"10.1016/j.mvr.2024.104753","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><div>To evaluate the performance of machine learning and then deep learning to detect a systemic scleroderma (SSc) landscape from the same set of nailfold capillaroscopy (NC) images from the French prospective multicenter observational study SCLEROCAP.</div></div><div><h3>Methods</h3><div>NC images from the first 100 SCLEROCAP patients were analyzed to assess the performance of machine learning and then deep learning in identifying the SSc landscape, the NC images having previously been independently and consensually labeled by expert clinicians. Images were divided into a training set (70 %) and a validation set (30 %). After features extraction from the NC images, we tested six classifiers (random forests (RF), support vector machine (SVM), logistic regression (LR), light gradient boosting (LGB), extreme gradient boosting (XGB), K-nearest neighbors (KNN)) on the training set with five different combinations of the images. The performance of each classifier was evaluated by the F1 score. In the deep learning section, we tested three pre-trained models from the TIMM library (ResNet-18, DenseNet-121 and VGG-16) on raw NC images after applying image augmentation methods.</div></div><div><h3>Results</h3><div>With machine learning, performance ranged from 0.60 to 0.73 for each variable, with Hu and Haralick moments being the most discriminating. Performance was highest with the RF, LGB and XGB models (F1 scores: 0.75–0.79). The highest score was obtained by combining all variables and using the LGB model (F1 score: 0.79 ± 0.05, <em>p</em> < 0.01). With deep learning, performance reached a minimum accuracy of 0.87. The best results were obtained with the DenseNet-121 model (accuracy 0.94 ± 0.02, F1 score 0.94 ± 0.02, AUC 0.95 ± 0.03) as compared to ResNet-18 (accuracy 0.87 ± 0.04, F1 score 0.85 ± 0.03, AUC 0.87 ± 0.04) and VGG-16 (accuracy 0.90 ± 0.03, F1 score 0.91 ± 0.02, AUC 0.91 ± 0.04).</div></div><div><h3>Conclusion</h3><div>By using machine learning and then deep learning on the same set of labeled NC images from the SCLEROCAP study, the highest performances to detect SSc landscape were obtained with deep learning and in particular DenseNet-121. This pre-trained model could therefore be used to automatically interpret NC images in case of suspected SSc. This result nevertheless needs to be confirmed on a larger number of NC images.</div></div>","PeriodicalId":18534,"journal":{"name":"Microvascular research","volume":"157 ","pages":"Article 104753"},"PeriodicalIF":2.9000,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Microvascular research","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S002628622400102X","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PERIPHERAL VASCULAR DISEASE","Score":null,"Total":0}
引用次数: 0
Abstract
Objective
To evaluate the performance of machine learning and then deep learning to detect a systemic scleroderma (SSc) landscape from the same set of nailfold capillaroscopy (NC) images from the French prospective multicenter observational study SCLEROCAP.
Methods
NC images from the first 100 SCLEROCAP patients were analyzed to assess the performance of machine learning and then deep learning in identifying the SSc landscape, the NC images having previously been independently and consensually labeled by expert clinicians. Images were divided into a training set (70 %) and a validation set (30 %). After features extraction from the NC images, we tested six classifiers (random forests (RF), support vector machine (SVM), logistic regression (LR), light gradient boosting (LGB), extreme gradient boosting (XGB), K-nearest neighbors (KNN)) on the training set with five different combinations of the images. The performance of each classifier was evaluated by the F1 score. In the deep learning section, we tested three pre-trained models from the TIMM library (ResNet-18, DenseNet-121 and VGG-16) on raw NC images after applying image augmentation methods.
Results
With machine learning, performance ranged from 0.60 to 0.73 for each variable, with Hu and Haralick moments being the most discriminating. Performance was highest with the RF, LGB and XGB models (F1 scores: 0.75–0.79). The highest score was obtained by combining all variables and using the LGB model (F1 score: 0.79 ± 0.05, p < 0.01). With deep learning, performance reached a minimum accuracy of 0.87. The best results were obtained with the DenseNet-121 model (accuracy 0.94 ± 0.02, F1 score 0.94 ± 0.02, AUC 0.95 ± 0.03) as compared to ResNet-18 (accuracy 0.87 ± 0.04, F1 score 0.85 ± 0.03, AUC 0.87 ± 0.04) and VGG-16 (accuracy 0.90 ± 0.03, F1 score 0.91 ± 0.02, AUC 0.91 ± 0.04).
Conclusion
By using machine learning and then deep learning on the same set of labeled NC images from the SCLEROCAP study, the highest performances to detect SSc landscape were obtained with deep learning and in particular DenseNet-121. This pre-trained model could therefore be used to automatically interpret NC images in case of suspected SSc. This result nevertheless needs to be confirmed on a larger number of NC images.
期刊介绍:
Microvascular Research is dedicated to the dissemination of fundamental information related to the microvascular field. Full-length articles presenting the results of original research and brief communications are featured.
Research Areas include:
• Angiogenesis
• Biochemistry
• Bioengineering
• Biomathematics
• Biophysics
• Cancer
• Circulatory homeostasis
• Comparative physiology
• Drug delivery
• Neuropharmacology
• Microvascular pathology
• Rheology
• Tissue Engineering.