Preprocessing of industrial process data with outlier detection and correction

Abstract

When constructing predictive models from process data using techniques such as neural networks, the validity of the data is very important. This paper presents some current methods of 'cleaning' data and proposes a structured method applied to a batch heat treatment application in the steel industry. The methodology highlights the use of expert knowledge throughout a project's evolution. The application of this data cleaning methodology to the heat treatment process is described, and a quantitative comparison is made of the performance of a neural network model by comparing the accuracy of its predictions before and after the correction of outlying points.