Formalizing big data processing lifecycles: Acquisition, serialization, aggregation, analysis, mining, knowledge representation, and information dissemination

Abstract

In today's e-Business environment, ERP, CRM, collaboration tools, and networked sensors may be characterized as data generators resources. Business Intelligence (BI) is a term that incorporates a range of analytical and decision support applications in business including data mining, decision support systems, knowledge management systems, and online analytical processing; processing data within these systems produce new data that are characterized to grow rapidly causing limitation problem of data management if handled by a Relational Database Management System (RDBMS) or statistical tools. Collectively these structured and unstructured data are referred to as Big Data. Successful and efficient handling of Big Data requires deployment of specific IT infrastructure components as well as adopting an emerging service model. In this research we introduce a conceptual model that abstracts the processing scheme of big data processing lifecycle. The model addresses the main phases of the lifecycle: data acquisition, data serialization, data aggregation, data analysis, data mining, knowledge representation, and information dissemination. The model is driven by projecting Service Oriented Architecture attributes to the building block of the lifecycle and adhering to the Lifecycle Modeling Language specification.