The Optimization of Big Data Platform Under the Internet of Things

Abstract

The development of the Internet of things(IOT) has produced huge diversity of data. In view of the need for massive data processing and application by the Internet of things, the big data service platform arises at the historic moment. Our paper mainly studies the optimization of the big data platform in the background of Internet of things, and combines the Internet of things with the big data platform–Spark. In this paper, we proposed an improved Spark job scheduling scheme based on the genetic and tabu search algorithm. By optimizing the job scheduling algorithm of Spark, it will provide the better technical support for data processing in the Internet of things.