MDMNet: Multi-dimensional multi-modal network to identify organ system limitation in cardiopulmonary exercise testing.

Background and objective: Cardiopulmonary exercise testing (CPET) serves as an integrative and comprehensive assessment tool for cardiorespiratory fitness. In this paper, we present a novel multi-dimensional multi-modal network (MDMNet) to identify functional limitation of organ systems via CPET, which is of great importance in clinical practice and yet a challenging task due to (1) the intricate intra-variable associations, and (2) the significant inter-individual variability.

Methods: The proposed model has three compelling characteristics. First, we employ a dedicated embedding strategy for CPET data to map raw inputs into the learned embedding space, facilitating the detection of latent features of physiological variables. Second, we devise a novel multi-dimensional feature extraction module to capture rich features of physiological inputs at different dimensions, which consists of a one-dimensional feature extraction branch unfolding both temporal and spatial patterns of the entire data, and a two-dimensional feature extraction branch based on Gramian Angular Field (GAF) encoding to reveal the complicated temporal correlation relationships between time points within a variable. Third, we integrate these techniques with clinically significant demographic information to establish our MDMNet incorporating multi-dimensional with multi-modal learning, thereby further addressing the issues of complex intra-variable associations and inter-individual variability simultaneously.

Results: We evaluated the proposed method on the publicly available CPET dataset, achieving AUC scores of 0.948, 0.949 and 0.931 for three tasks respectively.

Conclusions: The superiority of our method in discerning inter-individual differences was further demonstrated through partial least squares discriminant analysis, which holds significant potential for automated clinical application of CPET.