ETL-aware materialized view selection in semantic data stream warehouses

Abstract

For 25 years, several companies spent a lot of efforts and money in building warehouse (DW) applications for data analytics purposes. This technology contributes to the success stories of several companies. Nowadays, companies are looking for real-time analytics for data issued from fresh data sources and external resources as knowledge bases and linked open data. The traditional life-cycle of designing DW applications has to be revisited to meet this requirement. Note that this life-cycle is composed of several well-connected phases. Integrating this requirement will seriously impact all phases in charge of data which are: ETL (Extract, Transform, Load) and the physical design phase, in which physical optimization structures are selected to speed up OLAP queries. In this paper, we propose a Near Real Time Data Warehouse design (NRTDW) dealing with semantic data sources, with a particular focus on ETL and physical design phases. Firstly, we propose a dynamic materialized view selection method based on a workload of Sparql queries. Secondly, optimized algorithms are proposed to orchestrate the ETL flows considering the selected materialized views. Thirdly, an incremental view maintenance strategy recomputing only the graphs that involve the updated data sources is proposed. Finally, our findings are validated through an intensive experimentation using a detailed cost model on a real DBMS.