Big Data Mining: Managing the Costs of Data Mining

Abstract

The amount of data collected and stored in various industries has grown exponentially in the last decade. Data is collected and stored from industries consisting of large consumers such as telecommunications, banking or financial sectors. Further, given the advent of cloud computing and software availability in the cloud being cheaper, smaller industries are utilizing data storage for competitive advantage. Companies increasingly rely on analysis of huge amounts of data to gain a strategic advantage, improving on product quality and providing better services to their end users be it the employee, consumer or customer. A combination of statistical techniques and file management tools once sufficed for analyzing mounds of data. The costs of analysis are often charged out at very high rates for companies that require data analysis and the output is dependent very much on analyzing the correct attributes within large databases to ensure the data analyzed provides the relevant result. The most known technique or tools are the subject of the growing field of knowledge discovery in databases (KDD) [1]. Using business process data mapping (BPDM) to define the targeted data along with the process of knowledge discovery mapping in the database may provide a more targeted approach with much lest costs expended.