Learning and visualization of features using MC-DCNN for gait training considering physical individual differences

Several training methods have been developed to acquire motion information during real-time walking; these methods also feed the information back to the trainee. Trainees adjust their gait to ensure that the measured value approaches the target value, which may not always be suitable for each trainee. Therefore, we aim to develop a gait feedback training system that considers individual differences, classifies the gait of the trainee, and identifies adjustments for body parts and timing. A convolutional neural network (CNN) has a feature extraction function and is robust in terms of each feature position; therefore, it can be used to classify a gait as ideal or non-ideal. Additionally, when the gradient-weighted class activation mapping (Grad-CAM) is applied to the gait classification model, the output measures the influence degree contributed by the trainee’s each body part to the classification results. Thus, the trainee can visually determine the body parts that need to be adjusted through the use of the output. In this study, we focused on gaits related to stumbling. We measured the kinematics and kinetics data for participants and generated multivariate gait data, which were labeled as “gait rarely associated with stumbling” class or “gait frequently associated with stumbling” class using clustering with dynamic time warping. Next, the multichannel deep CNN (MC-DCNN) was used to learn the gait using the multivariate gait data and the corresponding classes. Finally, the data for verification were input into the MC-DCNN model, and we visualized the influence degrees of each place of the multivariate gait data for classification using Grad-CAM. The MC-DCNN model classified gaits with a high accuracy of 97.64±0.40%, and it learned the features that determine the thumb-to-ground distance. The output of the Grad-CAM indicated body parts, timing, and the relative strength of features that have an important effect on the thumb-to-ground distance. of the knee joint angle in the XZ plane angle and inverted ankle joint angle in the XZ plane for other data, and large ground reaction force X component. This indicates that the gait of data point 16 has insufficient flexion of the knee joint in mid-swing, abduction and external rotation of the hip joint and abduction of the ankle joint, and it shows “the circumduction gait.” When the mean of the “gait rarely associated with stumbling” class is presented to the trainee as a target value in the training, it is necessary for the participant of the data point 16 to adjust knee joint flexion angle, driving force of ground reaction force, and ankle joint angle in the XZ plane in the mid swing because the difference described in the Section 5.1 exists between the classes. However, in the heat map of the influence degree on the output score of each class by Grad-CAM, it is found that the knee joint angle in the YZ plane and the ground reaction force Y component affect the output score, but the influence degree on the ankle joint angle in the XZ plane is not observed. In addition, there is an influence on the output score in the trunk angle in the XZ plane, which was not large different between each mean of the classes, while there is no influence in the hip joint angle in the XZ plane and the knee joint angle in the XZ plane, which were not large different between each mean of the classes. These results indicate that the gait classification model for stumbling learned not only the mean of the classes but also the features that determine the thumb-to-ground distance from the shape of the waveform and the relationship among the variables. When attention is paid to “stumbling” as a disadvantage, the model judged that the abduction of the ankle joint, the abduction and external rotation of the hip joint, which are represented by the ankle joint angle in the XZ plane, hip joint angle in the XZ plane, and knee joint angle in the XZ plane are acceptable movements, and that the gait is suitable for individuals. The gait