Assessment of image quality on the diagnostic performance of clinicians and deep learning models: Cross-sectional comparative reader study.

IF 8.4 2区医学 Q1 DERMATOLOGY

Journal of the European Academy of Dermatology and Venereology Pub Date : 2024-12-10 DOI:10.1111/jdv.20462

A I Oloruntoba, M Asghari-Jafarabadi, M Sashindranath, Å Ingvar, N R Adler, C Vico-Alonso, L Niklasson, A L Caixinha, E Hiscutt, Z Holmes, K B Assersen, S Adamson, T Jegathees, T Bertelsen, V Velasco-Tamariz, T Helkkula, S Kristiansen, R Toholka, M S Goh, A Chamberlain, C McCormack, T Vestergaard, D Mehta, T D Nguyen, Z Ge, H P Soyer, V Mar

{"title":"Assessment of image quality on the diagnostic performance of clinicians and deep learning models: Cross-sectional comparative reader study.","authors":"A I Oloruntoba, M Asghari-Jafarabadi, M Sashindranath, Å Ingvar, N R Adler, C Vico-Alonso, L Niklasson, A L Caixinha, E Hiscutt, Z Holmes, K B Assersen, S Adamson, T Jegathees, T Bertelsen, V Velasco-Tamariz, T Helkkula, S Kristiansen, R Toholka, M S Goh, A Chamberlain, C McCormack, T Vestergaard, D Mehta, T D Nguyen, Z Ge, H P Soyer, V Mar","doi":"10.1111/jdv.20462","DOIUrl":null,"url":null,"abstract":"Background: Skin cancer is a prevalent and clinically significant condition, with early and accurate diagnosis being crucial for improved patient outcomes. Dermoscopy and artificial intelligence (AI) hold promise in enhancing diagnostic accuracy. However, the impact of image quality, particularly high dynamic range (HDR) conversion in smartphone images, on diagnostic performance remains poorly understood.Objective: This study aimed to investigate the effect of varying image qualities, including HDR-enhanced dermoscopic images, on the diagnostic capabilities of clinicians and a convolutional neural network (CNN) model.Methods: Eighteen dermatology clinicians assessed 303 images of 101 skin lesions that were categorized into three image quality groups: low quality (LQ), high quality (HQ) and enhanced quality (EQ) produced using HDR-style conversion. Clinicians participated in a two part reader study that required their diagnosis, management and confidence level for each image assessed.Results: In the binary classification of lesions, clinicians had the greatest diagnostic performance with HQ images, with sensitivity (77.3%; CI 69.1-85.5), specificity (63.1%; CI 53.7-72.5) and accuracy (70.2%; CI 61.3-79.1). For the multiclass classification, the overall performance was also best with HQ images, attaining the greatest specificity (91.9%; CI 83.2-95.0) and accuracy (51.5%; CI 48.4-54.7). Clinicians had a superior performance (median correct diagnoses) to the CNN model for the binary classification of LQ and EQ images, but their performance was comparable on the HQ images. However, in the multiclass classification, the CNN model significantly outperformed the clinicians on HQ images (p < 0.01).Conclusion: This study highlights the importance of image quality on the diagnostic performance of clinicians and deep learning models. This has significant implications for telehealth reporting and triage.","PeriodicalId":17351,"journal":{"name":"Journal of the European Academy of Dermatology and Venereology","volume":" ","pages":""},"PeriodicalIF":8.4000,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of the European Academy of Dermatology and Venereology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1111/jdv.20462","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"DERMATOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Skin cancer is a prevalent and clinically significant condition, with early and accurate diagnosis being crucial for improved patient outcomes. Dermoscopy and artificial intelligence (AI) hold promise in enhancing diagnostic accuracy. However, the impact of image quality, particularly high dynamic range (HDR) conversion in smartphone images, on diagnostic performance remains poorly understood.

Objective: This study aimed to investigate the effect of varying image qualities, including HDR-enhanced dermoscopic images, on the diagnostic capabilities of clinicians and a convolutional neural network (CNN) model.

Methods: Eighteen dermatology clinicians assessed 303 images of 101 skin lesions that were categorized into three image quality groups: low quality (LQ), high quality (HQ) and enhanced quality (EQ) produced using HDR-style conversion. Clinicians participated in a two part reader study that required their diagnosis, management and confidence level for each image assessed.

Results: In the binary classification of lesions, clinicians had the greatest diagnostic performance with HQ images, with sensitivity (77.3%; CI 69.1-85.5), specificity (63.1%; CI 53.7-72.5) and accuracy (70.2%; CI 61.3-79.1). For the multiclass classification, the overall performance was also best with HQ images, attaining the greatest specificity (91.9%; CI 83.2-95.0) and accuracy (51.5%; CI 48.4-54.7). Clinicians had a superior performance (median correct diagnoses) to the CNN model for the binary classification of LQ and EQ images, but their performance was comparable on the HQ images. However, in the multiclass classification, the CNN model significantly outperformed the clinicians on HQ images (p < 0.01).

Conclusion: This study highlights the importance of image quality on the diagnostic performance of clinicians and deep learning models. This has significant implications for telehealth reporting and triage.

查看原文本刊更多论文

对临床医生和深度学习模型的诊断性能的图像质量评估：横断面比较读者研究。

背景：皮肤癌是一种普遍且具有临床意义的疾病，早期准确诊断对于改善患者预后至关重要。皮肤镜检查和人工智能（AI）有望提高诊断准确性。然而，图像质量，特别是智能手机图像中的高动态范围（HDR）转换，对诊断性能的影响仍然知之甚少。目的：本研究旨在探讨不同图像质量（包括hdr增强皮肤镜图像）对临床医生和卷积神经网络（CNN）模型诊断能力的影响。方法：18名皮肤科临床医生对101个皮肤病变的303张图像进行了评估，这些图像被分为三个图像质量组：低质量（LQ）、高质量（HQ）和增强质量（EQ）。临床医生参与了一项分为两部分的读者研究，要求他们的诊断，管理和对每个图像评估的信心水平。结果：在病变的二元分类中，临床医生对HQ图像的诊断效能最高，敏感性为77.3%；CI 69.1-85.5)，特异性(63.1%；CI 53.7-72.5)和准确率(70.2%；可信区间61.3 - -79.1)。对于多类别分类，HQ图像的总体表现也最好，特异性最高(91.9%；CI 83.2-95.0)和准确率(51.5%；可信区间48.4 - -54.7)。临床医生在LQ和EQ图像的二值分类方面表现优于CNN模型（中位数正确诊断），但他们在HQ图像上的表现与CNN模型相当。然而，在多类分类中，CNN模型在HQ图像上的表现明显优于临床医生(p)。结论：本研究强调了图像质量对临床医生和深度学习模型诊断性能的重要性。这对远程医疗报告和分诊有重大影响。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of the European Academy of Dermatology and Venereology 医学-皮肤病学

CiteScore

10.70

自引率

8.70%

发文量

874

审稿时长

3-6 weeks

期刊介绍： The Journal of the European Academy of Dermatology and Venereology (JEADV) is a publication that focuses on dermatology and venereology. It covers various topics within these fields, including both clinical and basic science subjects. The journal publishes articles in different formats, such as editorials, review articles, practice articles, original papers, short reports, letters to the editor, features, and announcements from the European Academy of Dermatology and Venereology (EADV). The journal covers a wide range of keywords, including allergy, cancer, clinical medicine, cytokines, dermatology, drug reactions, hair disease, laser therapy, nail disease, oncology, skin cancer, skin disease, therapeutics, tumors, virus infections, and venereology. The JEADV is indexed and abstracted by various databases and resources, including Abstracts on Hygiene & Communicable Diseases, Academic Search, AgBiotech News & Information, Botanical Pesticides, CAB Abstracts®, Embase, Global Health, InfoTrac, Ingenta Select, MEDLINE/PubMed, Science Citation Index Expanded, and others.