Handling Data and Model Drift for World Application using Big Data

Abstract

It's still unclear how to effectively extract the information concealed inside vast and massive amounts of data. The problem of “idea drift” in stream data flows is one of the difficulties. Random walk is a common issue in data analytics where the statistical features of the characteristics and the categories they are intended for change with time, decreasing the accuracy of the trained model. There are numerous approaches that have been put out for bulk data mining. A new generation of data mining methods called stream mining updates the model in a single pass whenever fresh data is received. Because of its inherent adaptability, this one-pass mechanism may be properly capable than its predecessors of coping with idea drift in data streams. In this study, we assess a group of algorithms for mining the data streams using decision trees. The collection of rules which can be derived from the induced model is the benefit of decision tree learning. The predicate logics that represent the extracted rules can then be applied in a variety of decision-supporting applications. Even in the face of concept drift, the induced decision tree has to be precise and condensed. When dealing with concept-drift data, we evaluate how well three common incremental decision tree methods (random forest, isolation forest, and forest tree) perform. In the experiment, drift data from both synthetic and actual environments are employed. It is discovered that optimization method with big data technique i.e., MapReduce yields better outcomes.