The Assessment of Downstream Clinical Tasks in Cardiac Ultrasound Is Crucial for Segmentation Evaluation.

Abstract

Objective: To investigate the relationship between echocardiography segmentation accuracy and clinical examination indicators by comparing segmentation evaluation metrics (S-EM) with cardiac clinical examination indicators evaluation metrics (CCEI-EM).

Methods: Initially, four UNet-based segmentation models for 2D echocardiography were trained using the CAMUS dataset. Subsequently, 13 clinical examination indicators relevant to cardiac assessment were automatically extracted. Pixel accuracy (PA), dice similarity coefficient (DSC), intersection over union (IOU), and Hausdorff distance (HD) were utilized as S-EM to assess the segmentation outcomes. The deformation ratio between these clinical examination indicators and the ground truth was defined as CCEI-EM. Finally, statistical analyses were conducted to examine the correlations between S-EM and CCEI-EM.

Results: Statistically significant differences were detected between S-EM and cardiac structural indicator evaluation metrics (p < 0.05). However, weak correlations were identified for the left ventricular transverse diameter and the left ventricular long diameter with PA, DSC, and IOU (p < 0.05; r < 0.2). No statistically significant differences were found between cardiac functional evaluation metrics with PA, DSC, and IOU (p > 0.05); moreover, end-systolic volume demonstrated weak correlations with PA and DSC (p < 0.05; r < 0.2).

Conclusion: S-EM does not directly reflect the quality of cardiac clinical examination indicators derived from segmentation results. It is crucial to evaluate segmentation performance through downstream task-related clinical indicators.