Research on Oil Well Data Cleaning System

Abstract In the information age, with the continuous development of Internet technology, information data occupies every field of contemporary society. The development of the big data age makes these data more and more prominent. While users read the information they need from these massive data, data quality has also become a concern of users. A large number of data are preprocessed before data analysis, such as some duplicate values, missing values deal with inaccurate and other abnormal data, and filter the data through the data cleaning system to improve the standardization of the data, so as to improve the analysis efficiency of the data, reduce some unnecessary expenses, and save time and effort. The data cleaning system in this paper is implemented based on flash framework. Taking Python as the main language for data cleaning, technical cleaning and standard integration are carried out for some structural problems, duplication problems and missing problems of some different source data. Through the processing of abnormal data, the data quality and data analysis efficiency are greatly improved.