HDW: A High Performance Large Scale Data Warehouse

Abstract

As data warehouses grow in size, ensuring adequate database performance will be a big challenge. This paper presents a solution, called HDW, based on Google infrastructure such as GFS, Bigtable, MapReduce to build and manage a large scale distributed data warehouse for high performance OLAP analysis. In addition, HDW provides XMLA standard interface for front end applications. The results show that HDW achieves pretty good performance and high scalability, which has been demonstrated on at least 18 nodes with 36 cores.