Prior-driven refinement network for small organ-at-risk segmentation in head and neck cancer

Accurate segmentation of small organs-at-risk (OARs) in computed tomography (CT) images is crucial for radiotherapy treatment planning in head and neck cancer. However, the low soft tissue contrast, small spatial structures, and the limited training data pose significant challenges for automated segmentation methods. This paper proposes prior-driven refinement network (PRNet), a novel deep learning-based approach that leverages the foundation model’s general-purpose representations and domain-specific knowledge to tackle these challenges. PRNet builds upon the initial coarse segmentation and refines small organs by utilizing the coarse segmentation as prior knowledge. PRNet inherits its architecture from the Segment Anything Model (SAM) but incorporates a novel prior encoder and mask refinement transformer, enabling the fusion of domain-specific knowledge with SAM’s robust representations.The architecture of PRNet is inherited from the Segment Anything Model (SAM), with the addition of the prior encoder and the mask refinement transformer, allowing for the fusion of domain-specific knowledge with SAM’s robust representations. Experiments on three public datasets demonstrate PRNet’s superior performance, with average Dice scores of 75.14% ± 12.81%, 76.56% ± 12.90%, and 82.83 ± 13.49% respectively. These results represent improvements of 3.61%, 3.64%, and 5.14% over current state-of-the-art methods. Moreover, experiments on four diverse datasets demonstrate PRNet’s generalizability across different anatomical regions and imaging modalities, including liver tumors, myocardial pathologies, and thoracic organs. Our proposed method shows potential for improving clinical radiotherapy planning workflows and contributing to more precise treatment delivery in head and neck cancer patients.