BEMO: A Parsimonious Big Data Mining Methodology

Abstract

The Problem: Standardized processes are often followed to systematically conduct data mining projects. However while current models provide good descriptions, they are in need of updates given current Big Data challenges. Current data mining methods do not meet all requirements of businesses, in addition current methods are difficult to remember and do not cover all requisite steps. Given these limitations, usage of the traditional data mining process methods are fading in favor of independent data mining processes. What Was Done: BEMO (Business Opportunity, Exploration, Modeling, and Operationalization) is a standard parsimonious process developed for conducting data mining projects in a reusable and repeatable fashion in a Big Data environment. This model is vendor, technology, and industry agnostic. The process model is applied to a practical project example. Why this Work is Important: This manuscript allows a reusable and simplified model for data mining that can be applied to a variety of applications given a formalized and detailed process template. Given new technologies, Big Data and other developments a new data mining methodology is required to adequately meet these needs. The contribution of a parsimonious Big Data mining model also permits utilizing simpler models over complex models that can more efficiently generalize new problems.