Large Imbalance Data Classification Based on MapReduce for Traffic Accident Prediction

Abstract

In modern society, our everyday life has a close connection with traffic issues. One of the most burning issues is about predicting traffic accidents. Predicting accidents on the road can be achieved by classification analysis, a data mining procedure requiring enough data to build a learning model. Regarding building such a predicting system, there are several problems. It requires lots of hardware resources to collect traffic data and analyze it for predicting traffic accidents since the data is very huge. Furthermore, data related to traffic accidents is few comparing with data which is not related to them. The numbers of two types of data are imbalanced. The purpose of this paper is to build a predicting model that can resolve all these problems. This paper suggests using Hadoop framework to process and analyze big traffic data efficiently and a sampling method to resolve the problem of data imbalance. Based on this, the predicting system, first of all, preprocess traffic big data and analyzes it to create data for the learning system. The imbalance of created data is corrected by a sampling method. To improve predicting accuracy, corrected data is classified into several groups, to which classification analysis is applied. These analysis steps are processed by Hadoop framework.