A Comparative Evaluation of the Outlier Detection Methods

Abstract

In data mining, in order to calculate descriptive statistics and other statistical model parameters correctly, outliers should be identified and excluded from the data set before starting data analysis. This paper studied and compared the performance of model-based, density-based, clustering-based, angle-based, and isolation-based outlier detection methods used in data mining. ROC and AUC curves were used to compare the performances of outlier detection methods. A data set with a standard normal distribution and fit a logistic regression was simulated. To compare the methods, the data was modified by adding 30 outliers to the data set. The iForest algorithm was found to have higher prediction power than others. In addition, outliers were found in a real data set with the iForest algorithm, and the data set with outliers and without outliers were compared.