A longitudinally-consistent deep framework for joint subcortical segmentation and registration of infant brains

Abstract

Subcortical segmentation of infant brain MR images is fundamental for studying neurodevelopment but remains challenging due to rapid brain development, low tissue contrast, and ambiguous anatomical boundaries. Conventional age-specific methods typically segment longitudinal scans independently, which ignores crucial temporal guidance and frequently leads to longitudinally inconsistent results. To address these issues, we propose C2FSRnet, a longitudinally-consistent coarse-to-fine network for joint subcortical segmentation and registration. Inspired by the reciprocal benefits between these tasks, we develop a framework to simultaneously perform affine/deformable registration and segmentation of longitudinal scans. In the coarse stage, the registration and segmentation branches share a joint encoder, enabling them to assist each other in capturing more informative and generalized features. Specifically, we jointly input multiple longitudinal scans from each subject to train the framework to learn within-subject anatomical correspondences, providing explicit longitudinal guidance and ensuring consistency. Additionally, signed distance maps are incorporated as spatial contextual guidance into the fine-stage network in a multi-path manner to achieve refined segmentation and registration. We validated C2FSRnet on the UNC/UMN Baby Connectome Project (393 scans from 124 subjects) and the developing Human Connectome Project (20 scans from 10 subjects) datasets. Compared to 15 state-of-the-art methods, our framework consistently achieves superior accuracy and longitudinal consistency in both tasks, demonstrating its robust performance for characterizing early subcortical development in both healthy and clinical populations.