Ordinal isolation: An efficient and effective intelligent outlier detection algorithm

Abstract

Outlier detection plays important roles in intelligent cyber systems, especially for fault-tolerant and adaptive ones. Traditional algorithms always need to evaluate distances or densities, which are very time-consuming. On the increasingly urgent demand for real-time, during past years, various novel algorithms have been proposed. They are much faster, but less stable and accurate. To cope with these problems, with the core idea of ordinal optimization and the ‘few and different’ characteristics of outliers, by introducing the concept of outlier probability, we propose the ordinal isolation algorithm, which extracts outliers in terms of the order of being isolated in a recursive uniform data space partition process. It doesn't need any distance or density evaluating, and the complexity is reduced to O(n). Experiments show that, the CPU time of ordinal isolation increases linearly with linearly growing data sets. Furthermore, compared with recent iForest algorithm, ordinal isolation is about 30 times faster, with 20% to 30% improvement in accuracy, and especially is much more stable. Ordinal isolation also has good scalability, so it works well in high-dimensional data sets which have a huge number of instances and irrelevant attributes.