A Spark™ Based Client for Synchrophasor Data Stream Processing

Abstract

The SCADA based monitoring systems, having a very low sampling of one reading per 2-4 seconds is known to produce roughly 4.3 Tera Bytes (TiBs) of data annually. With synchrophasor technology, this will go up at least 100 times more as the rate of streaming is as high as 50/100 (60/120) Hz. Phasor data concentrators (PDCs) transmit byte streams encapsulating a comprehensive list of power system parameter including multiple phasor measurements, instantaneous frequency estimates, rate of change of frequency and several analog and digital quantities; this high volume and velocity of data makes it truly ‘Big Data’. This helps in making the power grid a lot more observable, enabling real-time monitoring of crucial grid events such as voltage stability, grid stress and transient oscillations. Synchrophasor technology uses the IEEE C37.118.2-2011™ Phasor Measurement Unit (PMU) / PDC communication protocol for data exchange which has no direct interface with any contemporary big data stream APIs or protocols. In this paper we propose a streaming interface in Apache Spark™, a popular big data platform, using Scala programming language, implementing a complete IEEE C37.118.2-2011™ client inside a stream receiver so that we can effortlessly receive synchrophasor data directly to Spark™ applications for real-time processing and archiving.