Processing big data with decision trees: A case study in large traffic data

Abstract

This paper provides a comparison of processing large traffic data by using decision trees. The experiment was tested in three different classifier tools that are very popular and are widely used in the community. These classifier tools are WEKA classifier, MoA (Massive Online Analysis) classifier, and SPARK MLib that runs on Hadoop infrastructure. We tested the traffic data using decision trees because it is one of the best methods for regressing the large data. The experiment results showed that the WEKA classifier fails to classify dataset with a large number of instance, wheras the MoA has successfully regress the dataset as a datastream. The SPARK MLib decision trees algorithm could also successfully resgress the traffic data quickly with a fairly good accuracy.