Automatic contour quality assurance using deep-learning based contours.

Abstract

Objective: Safe deployment of auto-contouring models requires the inclusion of automated QA. One such approach is to use two independent auto-contouring models and compare them geometrically for acceptability. This is not effective because geometric differences may not correlate with clinically significant errors. Herein, we investigated whether a two-contour QA system is improved by including dose in this comparison. Approach. VMAT plans were generated for 86 head and neck (H&N) and 50 cervical (GYN) cancer patients, using clinically-approved PTVs and auto-contour OARs from a primary auto-contouring model. Doses to the primary OARs were compared with doses to manually drawn and approved OARs ("the truth"). A difference in Dmean or Dmax ≥ 2 Gy was identified as a reporting error (Derror). A second, independent auto-contouring model was then used to contour the OARs (verification). The primary and verification auto-contouring models were compared geometrically (DSC, sDSC, HD95, MSD) and dosimetrically (Dmean, Dmax). The ability of comparison metrics between the two auto-contouring models to flag actual dosimetric errors (i.e. primary model compared with the truth) was investigated. A logistic regression model was used to predict Derror. The data was divided by disease site and into 50/50 stratified training and testing sets; k-fold cross validation was employed during training to avoid overfitting. H&N structures were further divided into size-specific groups to improve model performance and generalizability. Main Results. Including dose metrics in the logistic regression model to predict Derror, mean increased the performance in terms of ROC-AUC and AU-PRC in the test set for H&N small structures. For Derror, max, including dose metrics increased performance for H&N small structures, H&N medium structures, and GYN structures. Significance. In many instances, utilizing dose with geometric comparisons can improve the ability of a verification model to flag potential errors from a primary auto-contouring model. .