Research on the Acquisition and Processing of Multidimensional Power System Data Based on Web Crawlers

The digital transition of power companies has led to an explosion of all types of information. However, due to different standards and distinctive calibers, the information has not been effectively and deeply applied in the power systems. Electric power companies have a real demand for timely, accurate, and intelligent access to various information in the power industry. In this paper, we use the data acquisition technology, including Pandas, Numpy, BeautifulSoup, and other tool libraries, to build a data acquisition system that consists of an information data capture module, a data filtering module, and a data processing module. The related technologies of data storage and processing are also studied.