Analyzing network traffic data using Hive queries

Abstract

Billions of devices are connected together with internet to serve the communication. Network monitoring to detect various security threats has become crucial in any organization. In this paper, we analyze large amount of network traffic data using Hive database in Hadoop Distributed File System (HDFS) environment. Hive queries are developed to identify security threats. The results of queries are demonstrated and the Hive Client application is developed where all the queries can be integrated. An Apache Zeppelin Visualization Tool is also introduced which can provide more insights on the dataset.