A new approach for data cleaning process

Abstract

In this paper, we introduced a new approach called Effective Data Cleaning (EDC) is presented. The proposed EDC technique is aimed to identify the relevant and irrelevant instance from the large data set through the degree of the missing value, and it reconstructs the missed value in relevant instance through its closest instance within the instance set. The EDC technique is consist of two methods Identify Relevant Instance (IRI) and Reconstruct Missing Value (RMV). The IRI method is identifying the relevant and irrelevant instance belongs to the large instance set through the degree of the missing value of each instance in the instance set, and the RMV method can reconstruct the missing value in the relevant instance through its closest instance based on the distance metric. Experiment result shows, that the proposed EDC technique is simple and effective for identifying the relevant and irrelevant instance, and reconstruct the missing values in the relevant instance through the closest instance with higher similarity.