A method for sensitivity analysis of automatic contouring algorithms across different contrast weightings using synthetic magnetic resonance imaging

Background and purpose

A majority of institution-specific automatic magnetic resonance imaging (MRI)-based contouring algorithms utilize one contrast-weighting (i.e., T2-weighted), however their performance within this contrast-weighting (i.e., across different repetition time, TR, and echo time, TE) is under-investigated and poorly understood. The purpose of this study was to develop a method to evaluate the robustness of automatic contouring algorithms to varying MRI contrast-weightings.

Materials and methods

One healthy volunteer and one patient were scanned using the multi-delay multi-echo (MDME) scan on a 3T MRI. The parotid and submandibular glands in these subjects were contoured using an automatic contouring algorithm trained on T2-weighted MRIs. Ground truth consensus contours were created by two radiation oncology residents and one pre-resident physician. A total of 216 different TR and TE combinations were simulated across T1-, T2-, and PD-weighted contrast ranges using SyMRI. Comparisons between automatic contouring algorithm contours and the ground truth were determined using the Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95) with interobserver variability used as a threshold for clinical acceptance.

Results

Differences in the automatic contouring model’s performance were seen across the contrast-weighted regions. The range of discrepancy in DSC and HD95 exceeded 0.2 and 3.66 mm, respectively. In the T2-weighted contrast region, 100 %, 40 %, 24 %, and 57 % for the DSC in the left parotid, right parotid, left submandibular, and right submandibular gland, respectively, exceeded interobserver variability.

Conclusions

This study demonstrates the variable performance of MRI-based automatic contouring algorithms across varying TR and TE combinations even within the same contrast-weighting region (i.e., T2-weighted).