DERM-SUCCESS FDA Pivotal Study: A Multi-Reader Multi-Case Evaluation of Primary Care Physicians' Skin Cancer Detection Using AI-Enabled Elastic Scattering Spectroscopy.
Laura K Ferris, Erik Jaklitsch, Elizabeth V Seiverling, Thomas Agresta, Peggy Cyr, Laurie Caines, Na Wang, Daniel M Siegel
{"title":"DERM-SUCCESS FDA Pivotal Study: A Multi-Reader Multi-Case Evaluation of Primary Care Physicians' Skin Cancer Detection Using AI-Enabled Elastic Scattering Spectroscopy.","authors":"Laura K Ferris, Erik Jaklitsch, Elizabeth V Seiverling, Thomas Agresta, Peggy Cyr, Laurie Caines, Na Wang, Daniel M Siegel","doi":"10.1177/21501319251342106","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Elastic Scattering Spectroscopy (ESS), an optical tissue sampling technique, distinguishes between benign and malignant tissue in vivo without the need to perform a surgical biopsy. A handheld device that employs ESS enabled with an artificial intelligence algorithm was developed as an objective tool to aid primary care physicians (PCPs) in their management of lesions suspicious for skin cancer. The aim of this study was to assess and compare the diagnostic and management performance of PCPs with and without the use of the ESS device in detecting skin cancer.</p><p><strong>Methods: </strong>In this clinical utility study, 108 PCPs evaluated 100 skin lesion cases (50 aided with the device output and 50 unaided by the device). For each case, PCPs provided a diagnosis, management decision, and level of confidence in that decision initially without, and then subsequently with, device output. Sensitivity, specificity, AUC, and confidence in their assessment prior to, and then with, device output were compared.</p><p><strong>Results: </strong>With visual assessment assisted by device output, diagnostic sensitivity increased significantly from 71.1% to 81.7% (<i>P</i> = .0085) and referral sensitivity increased significantly from 82.0% to 91.4% (<i>P</i> = .0027) compared to visual assessment only. Device-aided diagnostic specificity decreased from 60.9% to 54.7% (<i>P</i> = .1896), and referral specificity decreased from 44.2% to 32.4% (<i>P</i> = .0256). Overall management performance (ie, AUC) also increased from 0.708 to 0.762, and increased from 0.567 to 0.682 for lesions which physicians reported low confidence in their unassisted management decision. Physicians reporting high confidence in their management assessment increased from 36.8% to 53.4%.</p><p><strong>Conclusion: </strong>Use of the ESS device output by PCPs significantly improved their diagnostic and management sensitivities as well as their overall management performance. The findings suggest the ESS device can improve PCP skin cancer detection and confidence in their skin lesion evaluation and management.</p>","PeriodicalId":46723,"journal":{"name":"Journal of Primary Care and Community Health","volume":"16 ","pages":"21501319251342106"},"PeriodicalIF":3.0000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12126654/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Primary Care and Community Health","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1177/21501319251342106","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/5/30 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PRIMARY HEALTH CARE","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Elastic Scattering Spectroscopy (ESS), an optical tissue sampling technique, distinguishes between benign and malignant tissue in vivo without the need to perform a surgical biopsy. A handheld device that employs ESS enabled with an artificial intelligence algorithm was developed as an objective tool to aid primary care physicians (PCPs) in their management of lesions suspicious for skin cancer. The aim of this study was to assess and compare the diagnostic and management performance of PCPs with and without the use of the ESS device in detecting skin cancer.
Methods: In this clinical utility study, 108 PCPs evaluated 100 skin lesion cases (50 aided with the device output and 50 unaided by the device). For each case, PCPs provided a diagnosis, management decision, and level of confidence in that decision initially without, and then subsequently with, device output. Sensitivity, specificity, AUC, and confidence in their assessment prior to, and then with, device output were compared.
Results: With visual assessment assisted by device output, diagnostic sensitivity increased significantly from 71.1% to 81.7% (P = .0085) and referral sensitivity increased significantly from 82.0% to 91.4% (P = .0027) compared to visual assessment only. Device-aided diagnostic specificity decreased from 60.9% to 54.7% (P = .1896), and referral specificity decreased from 44.2% to 32.4% (P = .0256). Overall management performance (ie, AUC) also increased from 0.708 to 0.762, and increased from 0.567 to 0.682 for lesions which physicians reported low confidence in their unassisted management decision. Physicians reporting high confidence in their management assessment increased from 36.8% to 53.4%.
Conclusion: Use of the ESS device output by PCPs significantly improved their diagnostic and management sensitivities as well as their overall management performance. The findings suggest the ESS device can improve PCP skin cancer detection and confidence in their skin lesion evaluation and management.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
