Turn Waste Into Wealth: On Efficient Clustering and Cleaning Over Dirty Data

Abstract

Dirty data commonly exist. Simply discarding a large number of inaccurate points (as noises) could greatly affect clustering results. We argue that dirty data can be repaired and utilized as strong supports in clustering. To this end, we study a novel problem of clustering and repairing over dirty data at the same time. Referring to the minimum change principle in data repairing, the objective is to find a minimum modification of inaccurate points such that the large amount of dirty data can enhance clustering. We show that the problem is np-hard and can be formulated as an integer linear programming (ilp) problem. A constant factor approximation algorithm gdorc is devised based on grid, with high efficiency. In experiments, gdorc has great repairing and clustering results with low time consumption. Empirical results demonstrate that both the clustering and cleaning accuracies can be improved by our approach of repairing and utilizing the dirty data in clustering.