Error awareness data mining

Real-world data mining applications often deal with low-quality information sources where data collection inaccuracy, device limitations, data transmission and discretization errors, or man-made perturbations frequently result in imprecise or vague data. Two common practices are to adopt either data cleansing to enhance data consistency or simply take noisy data as quality sources and feed them into the data mining algorithms. Either way may substantially sacrifice the mining performances. In this paper, we consider an error awareness data mining framework, which takes advantage of statistical error information (such as noise level and noise distribution) to improve data mining results. We assume such noise knowledge is available in advance, and propose a solution to incorporate it into the mining process. More specifically, we use noise knowledge to restore original data distributions, and then use the restored information to modify the model built from noise corrupted data. We present an Error Awareness Naive Bayes (EA_NB) classification algorithm, and provide extensive experimental comparisons to demonstrate the effectiveness of this effort.