Still Open Problems in Data Warehouse and Data Lake Research: extended abstract

Abstract

During recent years, we observe a widespread of new data sources, especially all types of social media and IoT devices, which produce huge data volumes, whose content ranges from fully structured to totally unstructured. All these types of data are commonly referred to as big data. They are typically described by the three most important characteristics, called 3V [1], namely: an extremely large volume, a variety of data models and structures (data representations), as well as a high velocity at which data are generated. We argue that out of these three Vs, the most challenging is variety [2]. Such data need to be integrated and transformed into a common representation, which is suitable for analysis, in a similar manner as traditional (mainly table-like) data.