Materialized View as a Service for Large-Scale House Log in Smart City

Abstract

Smart city provides various value-added services by collecting large-scale data from houses and infrastructures within a city. To use such large-scale raw data, individual applications usually take expensive computation effort and large processing time. To reduce the effort and time, we propose Materialized View as a Service (MVaaS). Using the MVaaS, each application can easily and dynamically construct its own materialized view, in which the raw data is cached in an appropriate format for the application. Once the view is constructed, the application can quickly access necessary data. In this paper, we design a framework of MVaaS specifically for large-scale house log, managed in our smart-city data platform Scallop4SC. In the framework, each application first specifies how the raw data should be filtered, grouped and aggregated. For a given data specification, MVaaS dynamically constructs a MapReduce batch program that converts the raw data into a desired view. The batch is then executed on Hadoop, and the resultant view is stored in HBase. We conduct an experimental evaluation to compare the response time between cases with and without the proposed MVaaS.