Deep Learning-Based Tumor Segmentation of Murine Magnetic Resonance Images of Prostate Cancer Patient-Derived Xenografts.

Background/objective: Longitudinal in vivo studies of murine xenograft models are widely utilized in oncology to study cancer biology and develop therapies. Magnetic resonance imaging (MRI) of these tumors is an invaluable tool for monitoring tumor growth and characterizing the tumors as well.

Methods: In this work, a pipeline for automating the segmentation of xenografts in mouse models was developed. T₂-weighted (T2-wt) MRI images from mice implanted with six different prostate cancer patient-derived xenografts (PDX) in the kidneys, liver, and tibia were used. The segmentation pipeline included a slice classifier to identify the slices that had tumors and subsequent training and validation using several U-Net-based segmentation architectures. Multiple combinations of the algorithm and training images for different sites were evaluated for inference quality.

Results and conclusions: The slice classifier network achieved 90% accuracy in identifying slices containing tumors. Among the various segmentation architectures tested, the dense residual recurrent U-Net achieved the highest performance in kidney tumors. When evaluated across the kidneys, tibia, and liver, this architecture performed the best when trained on all data as compared to training on only data from a single site (and inferring on a multi-site tumor images), achieving a Dice score of 0.924 across the test set.