A comprehensive deep learning model for motor phenotypes of Parkinson's disease using three-dimensional kinect V2 detectors

Background

Gait impairments are common for Parkinson’s disease (PD). With the development of artificial intelligence (AI) technology and three-dimensional Kinect V2 Detectors, it is possible to enable more accurate characterization of gait impairment. We develop a comprehensive prediction model by combining skeleton gait energy image with relative distance and angle for PD motor phenotypes.

Research question

Does the hybrid convolutional neural network-long short-term memory (CNN-LSTM) deep learning model improve diagnostic accuracy and outperform CNN or LSTM models in diagnosing different motor phenotypes of PD?

Method

We implemented and compared three deep learning architectures—CNN, LSTM, and a hybrid CNN-LSTM model. To mitigate class imbalance and enhance classification accuracy, the Synthetic Minority Oversampling Technique was applied. Feature relevance was determined using Random Forest (RF) and SHapley Additive exPlanations (SHAP), facilitating the identification of key predictors. Participants were stratified into three groups—healthy controls, non-postural instability, and gait disturbance (non-PIGD), and PIGD—based on mean scores from selected items of the Unified Parkinson’s Disease Rating Scale.

Results

The CNN–LSTM model demonstrated the highest predictive performance for PIGD classification during straight and turning walking in the off-medication state (AUC = 0.94 for both), followed by the CNN (AUC = 0.85 and 0.88) and LSTM models (AUC = 0.81 and 0.72). Moreover, the CNN–LSTM model achieved the highest classification accuracy across both on- and off-medication conditions. Using the DeLong test, we compared ROC curves of the CNN, LSTM, and hybrid CNN–LSTM models for PIGD classification across straight and turning walking tasks under both on- and off-medication conditions. The hybrid CNN–LSTM model consistently achieved significantly higher AUCs than the CNN and LSTM models in all settings.

Conclusion

Our study demonstrated that using a hybrid CNN-LSTM deep learning model in combination with RF and/or SHAP-based feature analysis, can achieve high classification performance.