Simultaneous perturbation stochastic approximation based neural networks for online learning

Abstract

This work presents a new application of simultaneous perturbation stochastic approximation (SPSA) for online learning and weight updates in multiple neural networks (SPSA-NN). A multi-agent system is implemented for the dynamic control of traffic signals in a complex traffic network with numerous intersections. Neural networks are used to approximate the optimal traffic signal control strategies for each agent and the parameters of these neural networks are updated online using an enhanced version of SPSA. Many simulation runs have been carried out to evaluate the performance of the SPSA-NN against an existing traffic signal control technique. Results show that the SPSA-NN based multi-agent system manages to outperform the existing technique. The mean delay of all vehicles has been reduced by 44% compared to the existing technique.