Agreement Between Nail Psoriasis Severity Index Scores by a Convolutional Neural Network and Dermatologists: A Retrospective Study at an Academic New York City Institution.
Jose W Ricardo, Rhiannon Miller, Matilde Iorizzo, Bianca M Piraccini, Michela Starace, Chander Grover, Dimitris Rigopoulos, Nilton Di Chiacchio, Nilton G Di Chiacchio, Hang Nguyen, Nga Nguyen, Zung Nguyen, Clifford Perlis, Jonathan Wolfe, Shari R Lipner
{"title":"Agreement Between Nail Psoriasis Severity Index Scores by a Convolutional Neural Network and Dermatologists: A Retrospective Study at an Academic New York City Institution.","authors":"Jose W Ricardo, Rhiannon Miller, Matilde Iorizzo, Bianca M Piraccini, Michela Starace, Chander Grover, Dimitris Rigopoulos, Nilton Di Chiacchio, Nilton G Di Chiacchio, Hang Nguyen, Nga Nguyen, Zung Nguyen, Clifford Perlis, Jonathan Wolfe, Shari R Lipner","doi":"10.1007/s40257-025-00934-y","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Nail psoriasis (NP) affects up to 90% and 86% of patients with cutaneous psoriasis and psoriatic arthritis, respectively, with a significant impact on quality-of-life. The Nail Psoriasis Severity Index (NAPSI) is infrequently used in clinical practice owing to its labor-intensive nature and variable interobserver reliability.</p><p><strong>Objective: </strong>The objective of this study was to assess performance and inter-reader agreement between artificial intelligence (AI)-determined NAPSI scores and dermatologist-assigned scores.</p><p><strong>Methods: </strong>This cross-sectional study used clinical images of psoriatic fingernails captured retrospectively at a specialized nail clinic in New York City. A convolutional neural network (CNN) model was trained and utilized for NAPSI classification of psoriatic fingernail clinical images, with seven dermatologist nail experts scoring identical images. The primary outcome was the interclass correlation coefficient (ICC), using a one-way analysis of variance (ANOVA) fixed effects model for the single-rater absolute agreement, between the average NAPSI score determined by the dermatologists and the AI.</p><p><strong>Results: </strong>In total, 240 images of psoriatic fingernails were included. The ICC for overall NAPSI, matrix (NAPSIm), and bed (NAPSIb) scores among the dermatologists were 0.43 (95% confidence interval [CI] 0.33-0.55), 0.56 (95% CI 0.46-0.67), and 0.53 (95% CI 0.43-0.65), respectively. Comparing the AI algorithm-assigned NAPSI, NAPSIm, and NAPSIb scores with the average dermatologist-assigned scores, ICCs were 0.81 (95% CI 0.74-0.86), 0.75 (95% CI 0.65-0.82), and 0.81 (95% CI 0.74-0.86), respectively.</p><p><strong>Conclusions: </strong>We found an excellent correlation between AI-derived NAPSI scores and dermatologist-assigned scores, underscoring the potential of CNNs to improve accuracy and reliability in NAPSI scoring. The limitations of this study include the small sample size, undetermined CNN diagnostic accuracy, incomplete data, and potential racial/ethnic minority group underrepresentation.</p>","PeriodicalId":7706,"journal":{"name":"American Journal of Clinical Dermatology","volume":" ","pages":""},"PeriodicalIF":8.6000,"publicationDate":"2025-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Journal of Clinical Dermatology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s40257-025-00934-y","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"DERMATOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Nail psoriasis (NP) affects up to 90% and 86% of patients with cutaneous psoriasis and psoriatic arthritis, respectively, with a significant impact on quality-of-life. The Nail Psoriasis Severity Index (NAPSI) is infrequently used in clinical practice owing to its labor-intensive nature and variable interobserver reliability.
Objective: The objective of this study was to assess performance and inter-reader agreement between artificial intelligence (AI)-determined NAPSI scores and dermatologist-assigned scores.
Methods: This cross-sectional study used clinical images of psoriatic fingernails captured retrospectively at a specialized nail clinic in New York City. A convolutional neural network (CNN) model was trained and utilized for NAPSI classification of psoriatic fingernail clinical images, with seven dermatologist nail experts scoring identical images. The primary outcome was the interclass correlation coefficient (ICC), using a one-way analysis of variance (ANOVA) fixed effects model for the single-rater absolute agreement, between the average NAPSI score determined by the dermatologists and the AI.
Results: In total, 240 images of psoriatic fingernails were included. The ICC for overall NAPSI, matrix (NAPSIm), and bed (NAPSIb) scores among the dermatologists were 0.43 (95% confidence interval [CI] 0.33-0.55), 0.56 (95% CI 0.46-0.67), and 0.53 (95% CI 0.43-0.65), respectively. Comparing the AI algorithm-assigned NAPSI, NAPSIm, and NAPSIb scores with the average dermatologist-assigned scores, ICCs were 0.81 (95% CI 0.74-0.86), 0.75 (95% CI 0.65-0.82), and 0.81 (95% CI 0.74-0.86), respectively.
Conclusions: We found an excellent correlation between AI-derived NAPSI scores and dermatologist-assigned scores, underscoring the potential of CNNs to improve accuracy and reliability in NAPSI scoring. The limitations of this study include the small sample size, undetermined CNN diagnostic accuracy, incomplete data, and potential racial/ethnic minority group underrepresentation.
背景：指甲牛皮癣（NP）分别影响高达90%和86%的皮肤牛皮癣和银屑病关节炎患者，对生活质量有显著影响。指甲银屑病严重程度指数（NAPSI）在临床实践中很少使用，因为它的劳动密集型性质和不同的观察者之间的可靠性。目的：本研究的目的是评估人工智能（AI）确定的NAPSI评分和皮肤科医生分配的评分之间的表现和读者间一致性。方法：本横断面研究采用回顾性拍摄的银屑病指甲临床图像在一个专门的指甲诊所在纽约市。训练卷积神经网络（CNN）模型并将其用于银屑病指甲临床图像的NAPSI分类，7名皮肤科指甲专家对相同的图像进行评分。主要结局是类间相关系数（ICC），使用由皮肤科医生和人工智能确定的平均NAPSI评分之间的单因素绝对一致性的单向方差分析（ANOVA）固定效应模型。结果：共纳入240张银屑病指甲图像。皮肤科医生的总体NAPSI、基质（NAPSIm）和床（NAPSIb）评分的ICC分别为0.43（95%可信区间[CI] 0.33-0.55）、0.56 （95% CI 0.46-0.67）和0.53 （95% CI 0.43-0.65）。将AI算法分配的NAPSI、NAPSIm和NAPSIb评分与皮肤科医生分配的平均评分进行比较，ICCs分别为0.81 （95% CI 0.74-0.86）、0.75 （95% CI 0.65-0.82）和0.81 （95% CI 0.74-0.86）。结论：我们发现ai衍生的NAPSI评分与皮肤科医生分配的评分之间存在良好的相关性，强调了cnn在提高NAPSI评分的准确性和可靠性方面的潜力。本研究的局限性包括样本量小、CNN诊断准确性不确定、数据不完整以及潜在的种族/少数民族代表性不足。
期刊介绍：
The American Journal of Clinical Dermatology is dedicated to evidence-based therapy and effective patient management in dermatology. It publishes critical review articles and clinically focused original research covering comprehensive aspects of dermatological conditions. The journal enhances visibility and educational value through features like Key Points summaries, plain language summaries, and various digital elements, ensuring accessibility and depth for a diverse readership.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
