Electrocardiogram Abnormality Classification From Summary Statistics With Patient-Level Validation and Process-Focused Evaluation

IF 0.9 4区医学 Q4 CARDIAC & CARDIOVASCULAR SYSTEMS

Annals of Noninvasive Electrocardiology Pub Date : 2026-08-11 DOI:10.1111/anec.70228

Sungjoon Hong, Christina Hartnett, Michael Ruane, Jason Tagliarino, Michael Nizich, M. Toma

{"title":"Electrocardiogram Abnormality Classification From Summary Statistics With Patient-Level Validation and Process-Focused Evaluation","authors":"Sungjoon Hong, Christina Hartnett, Michael Ruane, Jason Tagliarino, Michael Nizich, M. Toma","doi":"10.1111/anec.70228","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Machine learning (ML) applications in clinical medicine are vulnerable to data leakage, particularly temporal leakage from post-diagnostic features and patient-level leakage from improper partitioning, compromising electrocardiogram (ECG) abnormality detection systems. This study addresses these vulnerabilities through patient-level data splitting and systematic evaluation across multiple classification scenarios.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>ECG data from 4419 observations representing 2180 unique patients were analyzed using Random Under-Sampling Boosting (RUSBoost). Three binary classification scenarios were defined: Scenario 1 (Abnormal vs. Normal, excluding borderline), Scenario 2 (Abnormal + Borderline vs. Normal), and Scenario 3 (Abnormal vs. Normal + Borderline). Patient-level stratified partitioning (60:20:20) prevented information leakage. Models were evaluated using accuracy, sensitivity, specificity, F1-score, learning curves, and precision-recall curves across all partitions.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Scenario 1 demonstrated optimal generalization with test accuracy of 70.79%, sensitivity of 60.30%, specificity of 86.50%, and F1-score of 0.712, exhibiting monotonic decline across partitions (73.98%→71.50%→70.79%). Scenario 2 achieved 65.26% test accuracy with greater degradation (6.17 percentage points), reflecting increased difficulty when borderline cases were grouped with abnormal findings. Scenario 3 exhibited problematic non-monotonic patterns (71.84%→67.71%→69.36%) with premature convergence, indicating fundamental generalization challenges despite near-balanced classes (1.07:1 ratio).</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>Aggregate metrics alone inadequately support clinical deployment decisions. Medical AI evaluation must examine learning dynamics, generalization stability, and precision-recall calibration across partitions. Process-focused standards assessing monotonic decline, convergence characteristics, and calibration stability are essential for reliable ECG abnormality detection deployment.</p>\n </section>\n </div>","PeriodicalId":8074,"journal":{"name":"Annals of Noninvasive Electrocardiology","volume":"31 5","pages":""},"PeriodicalIF":0.9000,"publicationDate":"2026-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/anec.70228","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Noninvasive Electrocardiology","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/anec.70228","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"CARDIAC & CARDIOVASCULAR SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Machine learning (ML) applications in clinical medicine are vulnerable to data leakage, particularly temporal leakage from post-diagnostic features and patient-level leakage from improper partitioning, compromising electrocardiogram (ECG) abnormality detection systems. This study addresses these vulnerabilities through patient-level data splitting and systematic evaluation across multiple classification scenarios.

Methods

ECG data from 4419 observations representing 2180 unique patients were analyzed using Random Under-Sampling Boosting (RUSBoost). Three binary classification scenarios were defined: Scenario 1 (Abnormal vs. Normal, excluding borderline), Scenario 2 (Abnormal + Borderline vs. Normal), and Scenario 3 (Abnormal vs. Normal + Borderline). Patient-level stratified partitioning (60:20:20) prevented information leakage. Models were evaluated using accuracy, sensitivity, specificity, F1-score, learning curves, and precision-recall curves across all partitions.

Results

Scenario 1 demonstrated optimal generalization with test accuracy of 70.79%, sensitivity of 60.30%, specificity of 86.50%, and F1-score of 0.712, exhibiting monotonic decline across partitions (73.98%→71.50%→70.79%). Scenario 2 achieved 65.26% test accuracy with greater degradation (6.17 percentage points), reflecting increased difficulty when borderline cases were grouped with abnormal findings. Scenario 3 exhibited problematic non-monotonic patterns (71.84%→67.71%→69.36%) with premature convergence, indicating fundamental generalization challenges despite near-balanced classes (1.07:1 ratio).

Conclusions

Aggregate metrics alone inadequately support clinical deployment decisions. Medical AI evaluation must examine learning dynamics, generalization stability, and precision-recall calibration across partitions. Process-focused standards assessing monotonic decline, convergence characteristics, and calibration stability are essential for reliable ECG abnormality detection deployment.

查看原文本刊更多论文

心电图异常分类从汇总统计与患者水平的验证和过程为重点的评估。

背景：机器学习（ML）在临床医学中的应用很容易受到数据泄露的影响，特别是诊断后特征的时间泄漏和患者层面的不当划分泄漏，危及心电图（ECG）异常检测系统。本研究通过患者级别的数据拆分和跨多个分类场景的系统评估来解决这些漏洞。方法：采用随机欠采样增强（RUSBoost）对来自2180例独特患者的4419例观察的心电图数据进行分析。定义了三种二元分类场景：场景1（异常vs.正常，不包括边缘），场景2（异常+边缘vs.正常），场景3（异常vs.正常+边缘）。患者级分层分区（60:20:20）防止信息泄露。使用所有分区的准确性、灵敏度、特异性、f1评分、学习曲线和精确召回率曲线对模型进行评估。结果：场景1具有最佳的泛化效果，测试准确率为70.79%，灵敏度为60.30%，特异性为86.50%，f1评分为0.712，呈现出跨分区的单调下降（73.98%→71.50%→70.79%）。场景2实现了65.26%的测试准确率，但下降幅度更大（6.17个百分点），这反映了将边缘病例与异常结果分组时的难度增加。场景3显示出有问题的非单调模式（71.84%→67.71%→69.36%）和过早收敛，表明尽管类接近平衡（1.07:1），但基本的泛化挑战。结论：单独的综合指标不足以支持临床部署决策。医疗人工智能评估必须检查学习动态、泛化稳定性和跨分区的查准率校准。以过程为中心的标准评估单调下降、收敛特性和校准稳定性对于可靠的心电异常检测部署至关重要。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Annals of Noninvasive Electrocardiology 医学-心血管系统

CiteScore

3.40

自引率

0.00%

发文量

审稿时长

6-12 weeks

期刊介绍： The ANNALS OF NONINVASIVE ELECTROCARDIOLOGY (A.N.E) is an online only journal that incorporates ongoing advances in the clinical application and technology of traditional and new ECG-based techniques in the diagnosis and treatment of cardiac patients. ANE is the first journal in an evolving subspecialty that incorporates ongoing advances in the clinical application and technology of traditional and new ECG-based techniques in the diagnosis and treatment of cardiac patients. The publication includes topics related to 12-lead, exercise and high-resolution electrocardiography, arrhythmias, ischemia, repolarization phenomena, heart rate variability, circadian rhythms, bioengineering technology, signal-averaged ECGs, T-wave alternans and automatic external defibrillation. ANE publishes peer-reviewed articles of interest to clinicians and researchers in the field of noninvasive electrocardiology. Original research, clinical studies, state-of-the-art reviews, case reports, technical notes, and letters to the editors will be published to meet future demands in this field.