Osteo-fusion: A multimodal decision-chaining approach for automated knee osteoarthritis detection & severity classification

Abstract

Purpose

Knee Osteoarthritis (KOA) is a degenerative joint condition that affects the knee, caused by gradual deterioration of cartilage. Applying Machine Learning (ML) principles to the Medical Imaging (MI) data, related to KOA, has the ability to significantly improve automated disease identification and severity analysis.

Materials and methods

This study proposes a novel predictive classifier model, named as Osteo-Fusion, based on a Decision chaining approach, which combines the strengths of different modalities, such as X-ray and Gait to enable efficient automated diagnosis and severity classification of KOA. The proposed technique integrates the advantages of Transfer learning and Fusion learning to enhance the efficiency of the automated diagnostic process. The proposed technique employs a two-stage decision-chaining approach based on decision-level fusion.

Results

The proposed model achieved higher accuracy and precision values across both the X-ray and Gait classification tasks. The optimized VGG-16 model achieved 98.5% training accuracy and 96% validation accuracy on the X-ray dataset. The optimized VGG-16 model demonstrated strong performance on the gait dataset as well for severity classification, by obtaining 99% Training accuracy and 97% Validation accuracy,achieving an overall accuracy of 98%, precision of 0.99, recall of 0.97, and an F1-score of 0.98 across various performance metrics for severity classification. The proposed decision-chaining approach, which integrates structural and functional assessments for KOA classification, achieved an overall accuracy of 85% and a weighted F1-score of 0.8325 on the testing dataset. Grad-CAM visualizations are used to enhance interpretability by highlighting the regions influencing the model’s decisions.

Conclusion

The proposed model leverages the complementary strengths of multiple modalities, X-ray for structural assessment and gait analysis for functional evaluation, resulting in improved overall performance in automated disease diagnosis and severity classification. The accuracy achieved by optimized VGG-16 on X-ray and Gait is significantly higher as compared to the existing systems. The simulated decision-chaining system shows strong performance in identifying Moderate and Severe cases.