Knee-cartilage segmentation from MR images using Multi-view Hypergraph Convolutional Neural Networks

Abstract

Leveraging the increased capacities of hypergraphs to model complex data structures, we propose in this article the Multi-view Hyper-Graph Convolutional Network (MVHGCN) to yield automated knee-joint cartilage segmentations from MRIs. The main properties of our approach are presented as follows: 1) Node features are obtained from multi-view (MV) acquisitions, corresponding to different feature extractors or image modalities. 2) Node embeddings are generated using a distributive MV convolution scheme which combines the various view-specific convolutions. These results are aggregated via an attention-based fusion module to automatically learn the weights of the different views. 3) Our model integrates both local and global level learning, simultaneously. Local hypergraph convolutions explore the relationships across the spatially aligned node libraries, while global hypergraph convolutions search for global affinities between nodes located at different positions within the image. 4) We propose two different blending schemes to combine local and global convolutions, namely, the cross-talk (CT) and the collaborative (COL) blending units, respectively. Using these units as building blocks, we construct the MVHGCN model, a deep network with enhanced feature representation and learning capabilities. The suggested segmentation method is evaluated on the publicly available Osteoarthritis Initiative (OAI) cohort. Specifically, we have designed a thorough experimental setup, including parameter sensitivity analysis and comparative results against a series of existing traditional methods, deep CNN models, and graph convolutional networks. The results show that MVHGCN outperforms the competing methods, achieving an overall cartilage segmentation score of \(\mathcal {DSC} = 95.81\%\) and \(\mathcal {DSC} = 96.33\%\), for the CT and the COL blending, respectively.