Modelling Genomics Datasets to Optimize Its Usage Patterns for Efficient Analytics

Abstract

Data intensive computing requires fast access to the input data. Currently in an HPC environment, the computation part is managed by the job scheduler, whereas data access is unmanaged and reply completely on the underlying file system. The job scheduler tracks system parameters such as RAM, CPU and utilize this for efficient computation. However, it is the responsibility of the developer to take care of the data related attributes and their traffic over the network. To solve this issue, we suggest data models for genomics data which can be used to guide the computation engine from data management perspective in addition to the usual system and computation parameters -- We will demonstrated that enabling the job scheduler with info about the data is an important steps towards efficient fully automated data analysis.