Jonathan Hourmozdi MD, MSAI, MA , Nicholas Easton MSAI , Simon Benigeri MSAI , James D. Thomas MD , Akhil Narang MD , David Ouyang MD , Grant Duffy BS , Ross Upton PhD , Will Hawkes PhD , Ashley Akerman PhD , Ike Okwuosa MD , Adrienne Kline MD, PhD , Abel N. Kho MD , Yuan Luo PhD , Sanjiv J. Shah MD , Faraz S. Ahmad MD, MS
{"title":"Evaluating the Performance and Potential Bias of Predictive Models for Detection of Transthyretin Cardiac Amyloidosis","authors":"Jonathan Hourmozdi MD, MSAI, MA , Nicholas Easton MSAI , Simon Benigeri MSAI , James D. Thomas MD , Akhil Narang MD , David Ouyang MD , Grant Duffy BS , Ross Upton PhD , Will Hawkes PhD , Ashley Akerman PhD , Ike Okwuosa MD , Adrienne Kline MD, PhD , Abel N. Kho MD , Yuan Luo PhD , Sanjiv J. Shah MD , Faraz S. Ahmad MD, MS","doi":"10.1016/j.jacadv.2025.101901","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Delays in the diagnosis of transthyretin amyloid cardiomyopathy (ATTR-CM) contribute to the significant morbidity of the condition, especially in the era of disease-modifying therapies. Screening for ATTR-CM with artificial intelligence and other algorithms may improve timely diagnosis, but these algorithms have not been directly compared.</div></div><div><h3>Objectives</h3><div>The aim of this study was to compare the performance of 4 algorithms for ATTR-CM detection in a heart failure population and assess the risk for harms due to model bias.</div></div><div><h3>Methods</h3><div>We identified patients in an integrated health system from 2010 to 2022 with ATTR-CM and age- and sex-matched them to controls with heart failure to target 5% prevalence. We compared the performance of a claims-based random forest model (Huda et al model), a regression-based score (Mayo ATTR-CM), and 2 deep learning echo models (EchoNet-LVH and EchoGo Amyloidosis). We evaluated for bias using standard fairness metrics.</div></div><div><h3>Results</h3><div>The analytical cohort included 176 confirmed cases of ATTR-CM and 3,192 control patients with 79.2% self-identified as White and 9.0% as Black. The Huda et al model performed poorly (AUC: 0.49). Both deep learning echo models had a higher AUC when compared to the Mayo ATTR-CM Score (EchoNet-LVH 0.88; EchoGo Amyloidosis 0.92; Mayo ATTR-CM Score 0.79; DeLong <em>P</em> < 0.001 for both). Bias auditing met fairness criteria for <em>equal opportunity</em> among patients who identified as Black.</div></div><div><h3>Conclusions</h3><div>Deep learning, echo-based models to detect ATTR-CM demonstrated best overall discrimination when compared to 2 other models in external validation with low risk of harms due to racial bias.</div></div>","PeriodicalId":73527,"journal":{"name":"JACC advances","volume":"4 8","pages":"Article 101901"},"PeriodicalIF":0.0000,"publicationDate":"2025-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"JACC advances","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2772963X25003217","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Background
Delays in the diagnosis of transthyretin amyloid cardiomyopathy (ATTR-CM) contribute to the significant morbidity of the condition, especially in the era of disease-modifying therapies. Screening for ATTR-CM with artificial intelligence and other algorithms may improve timely diagnosis, but these algorithms have not been directly compared.
Objectives
The aim of this study was to compare the performance of 4 algorithms for ATTR-CM detection in a heart failure population and assess the risk for harms due to model bias.
Methods
We identified patients in an integrated health system from 2010 to 2022 with ATTR-CM and age- and sex-matched them to controls with heart failure to target 5% prevalence. We compared the performance of a claims-based random forest model (Huda et al model), a regression-based score (Mayo ATTR-CM), and 2 deep learning echo models (EchoNet-LVH and EchoGo Amyloidosis). We evaluated for bias using standard fairness metrics.
Results
The analytical cohort included 176 confirmed cases of ATTR-CM and 3,192 control patients with 79.2% self-identified as White and 9.0% as Black. The Huda et al model performed poorly (AUC: 0.49). Both deep learning echo models had a higher AUC when compared to the Mayo ATTR-CM Score (EchoNet-LVH 0.88; EchoGo Amyloidosis 0.92; Mayo ATTR-CM Score 0.79; DeLong P < 0.001 for both). Bias auditing met fairness criteria for equal opportunity among patients who identified as Black.
Conclusions
Deep learning, echo-based models to detect ATTR-CM demonstrated best overall discrimination when compared to 2 other models in external validation with low risk of harms due to racial bias.