Enhancing patient rehabilitation predictions with a hybrid anomaly detection model: Density-based clustering and interquartile range methods

Abstract

In recent years, there has been a concerted effort to improve anomaly detection techniques, particularly in the context of high-dimensional, distributed clinical data. Analysing patient data within clinical settings reveals a pronounced focus on refining diagnostic accuracy, personalising treatment plans, and optimising resource allocation to enhance clinical outcomes. Nonetheless, this domain faces unique challenges, such as irregular data collection, inconsistent data quality, and patient-specific structural variations. This paper proposed a novel hybrid approach that integrates heuristic and stochastic methods for anomaly detection in patient clinical data to address these challenges. The strategy combines HPO-based optimal Density-Based Spatial Clustering of Applications with Noise for clustering patient exercise data, facilitating efficient anomaly identification. Subsequently, a stochastic method based on the Interquartile Range filters unreliable data points, ensuring that medical tools and professionals receive only the most pertinent and accurate information. The primary objective of this study is to equip healthcare professionals and researchers with a robust tool for managing extensive, high-dimensional clinical datasets, enabling effective isolation and removal of aberrant data points. Furthermore, a sophisticated regression model has been developed using Automated Machine Learning (AutoML) to assess the impact of the ensemble abnormal pattern detection approach. Various statistical error estimation techniques validate the efficacy of the hybrid approach alongside AutoML. Experimental results show that implementing this innovative hybrid model on patient rehabilitation data leads to a notable enhancement in AutoML performance, with an average improvement of 0.041 in the $R^2$ score, surpassing the effectiveness of traditional regression models.