A Disk-Based Algorithm for Fast Outlier Detection in Large Datasets

Abstract

Outlier Detection is an important research issue in Data Mining,the number of cells in the cell-based disk algorithm increases exponentially.The performance of this algorithm will decrease dramatically with the increasing of the number of cell and data points.Further analysis finds that there are many empty cells that are useless to outlier detection.To overcome these shortcomings,this paper apply the CD-Tree to index non-empty cells,and cluster technique is adopted to store the data objects in the same cell into linked disk pages.Some experiments are made to test the performance of the proposed algorithms.The experimental results show that the performance of the CD-Tree based and cluster-based disk algorithms are outperformed that of the cell-based disk algorithm,and the number of dimensions processed by the proposed algorithms is higher than that of the old one.