Real-Time Taxi Demand Prediction using data from the web

In transportation, nature, economy, environment, and many other settings, there are multiple simultaneous phenomena happening that are of interest to model and predict. Over the last few years, the traffic data that we have at our disposal have significantly increased, and we have truly entered the era of big data for transportation. Most existing traffic flow prediction methods mainly focus on capturing recurrent mobility trends that relate to habitual/routine behaviour, and on exploiting short-term correlations with recent observation patterns. However, valuable information that is often available in the form of unstructured data is neglected when attempting to improve forecasting results. In this paper, we explore time-series data and textual information combinations using machine learning techniques in the context of creating a prediction model that is able to capture in real-time future stressful situations of the studied transportation system. Using publicly available taxi data from New York, we empirically show that the proposed models are able to significantly reduce the error in the forecasts. The final mean absolute error (MAE) of our predictions is decreased by 19.5% for a three months testing period and by 57% if we focus only on event periods.