Qualitative evaluation of automatic liver segmentation in computed tomography images for clinical use in radiation therapy

Purpose

Segmentation of target volumes and organs at risk on computed tomography (CT) images constitutes an important step in the radiotherapy workflow. Artificial intelligence-based methods have significantly improved organ segmentation in medical images. Automatic segmentations are frequently evaluated using geometric metrics. Before a clinical implementation in the radiotherapy workflow, automatic segmentations must also be evaluated by clinicians. The aim of this study was to investigate the correlation between geometric metrics used for segmentation evaluation and the assessment performed by clinicians.

Materials and Methods

In this study, we used the U-Net model to segment the liver in CT images from a publicly available dataset. The model's performance was evaluated using two geometric metrics: the Dice similarity coefficient and the Hausdorff distance. Additionally, a qualitative evaluation was performed by clinicians who reviewed the automatic segmentations to rate their clinical acceptability for use in the radiotherapy workflow. The correlation between the geometric metrics and the clinicians’ evaluations was studied.

Results

The results showed that while the Dice coefficient and Hausdorff distance are reliable indicators of segmentation accuracy, they do not always align with clinician segmentation. In some cases, segmentations with high Dice scores still required clinician corrections before clinical use in the radiotherapy workflow.

Conclusion

This study highlights the need for more comprehensive evaluation metrics beyond geometric measures to assess the clinical acceptability of artificial intelligence-based segmentation. Although the deep learning model provided promising segmentation results, the present study shows that standardized validation methodologies are crucial for ensuring the clinical viability of automatic segmentation systems.