A theory-informed multivariate causal framework for trustworthy short-term urban traffic forecasting

Traffic forecasting using Deep Learning has been a remarkably active and innovative research field during the last decades. However, there are still several barriers to real-world, large-scale implementation of Deep Learning forecasting models, including their data requirements, limited explainability and low efficiency. In this paper, we propose a novel theory-driven framework that is based on a Granger causality-inspired feature selection method and a multitask LSTM to jointly predict two traffic variables. Traffic flow theory intuition is induced in the training process by an enhanced Traffic Flow Theory-Informed loss function (TFTI loss), which includes the divergence of the joint prediction of two traffic variables from the actual fundamental diagram of the corresponding location. The theory-informed, Granger causal, multitask LSTM is trained for one step ahead volume and speed forecasting using loop detector data coming from the extended Athens road network (Greece). Findings indicate that the models trained using the TFTI loss and a reduced input space, which includes only causal information, achieve a significantly improved performance, compared to the models using the classic Mean Squared Error loss function. Moreover, we introduce a dedicated trustworthiness evaluation framework that indicates that the proposed approach enhances the trustworthiness of the predictions, as well as the models’ transparency and resilience to noisy data.