Automatic sequence identification in multicentric prostate multiparametric MRI datasets for clinical machine-learning.

Abstract

Objectives: To present an accurate machine-learning (ML) method and knowledge-based heuristics for automatic sequence-type identification in multi-centric multiparametric MRI (mpMRI) datasets for prostate cancer (PCa) ML.

Methods: Retrospective prostate mpMRI studies were classified into 5 series types-T2-weighted (T2W), diffusion-weighted images (DWI), apparent diffusion coefficients (ADC), dynamic contrast-enhanced (DCE) and other series types (others). Metadata was processed for all series and two models were trained (XGBoost after custom categorical tokenization and CatBoost with raw categorical data) using 5-fold cross-validation (CV) with different data fractions for learning curve analyses. For validation, two test sets-hold-out test set and temporal split-were used. A leave-one-group-out (LOGO) CV analysis was performed with centres as groups to understand the effect of dataset-specific data.

Results: 4045 studies (31,053 series) and 1004 studies (7891 series) from 11 centres were used to train and test series identification models, respectively. Test F1-scores were consistently above 0.95 (CatBoost) and 0.97 (XGBoost). Learning curves demonstrate learning saturation, while temporal validation shows model remain capable of correctly identifying all T2W/DWI/ADC triplets. However, optimal performance requires centre-specific data-controlling for model and used feature sets when comparing CV with LOGOCV, F1-score dropped for T2W, DCE and others (-0.146, -0.181 and -0.179, respectively), with larger performance decreases for CatBoost (-0.265). Finally, we delineate heuristics to assist researchers in series classification for PCa mpMRI datasets.

Conclusions: Automatic series-type identification is feasible and can enable automated data curation. However, dataset-specific data should be included to achieve optimal performance.

Critical relevance statement: Organising large collections of data is time-consuming but necessary to train clinical machine-learning models. To address this, we outline and validate an automatic series identification method that can facilitate this process. Finally, we outline a set of metadata-based heuristics that can be used to further automate series-type identification.

Key points: Multi-centric prostate MRI studies were used for sequence annotation model training. Automatic sequence annotation requires few instances and generalises temporally. Sequence annotation, necessary for clinical AI model training, can be performed automatically.