COMPARATIVE EVALUATION OF THE EFFICIENCY OF REINFORCEMENT MACHINE LEARNING ALGORITHMS AND COMPARATIVE EVALUATION OF TRANSPORT PROCESSES IN THE ENVIRONMENT ANYLOGIC

Abstract

The task of increasing the throughput of sections of the transport network of the metropolis with the existing infrastructure is solved by means of automated traffic control systems, the purpose of which is to form control actions on the objects of the transport system in real time. An adequate response to load changes in the transport network is implemented by controlling traffic light objects using built-in adaptive algorithms, among which artificial intelligence technologies, in particular, neural networks, are increasingly being used. The noted approaches compete with the widely used optimization-based adaptation method (phase duration, displacement, etc.). The proposed article examines the issue of comparing the efficiency of an optimization algorithm with reinforcement machine learning algorithms by the criterion of the time spent by vehicles in the intersection system in the Anylogic simulation environment. This study will help determine a more efficient solution to the problem of setting the duration of traffic light control phases. It was shown that in the reinforcement learning algorithm, the ability to adapt to input intensities within the schedule is higher compared to the optimization algorithm. However, the reinforcement algorithm is more sensitive to the type of schedule than the optimization algorithm, which outperformed the latter by about 20% at the optimal point for the weekday schedule. The advantage of the reinforcement algorithm was more pronounced on the 2nd schedule, which features a tendency to increase traffic intensity, namely: 61% compared to the optimization and 70% compared to the base setting. It turned out to be practically insensitive to the “detuning” of the input data relative to the optimal policy when changing the intensity levels within this schedule. Thus, it was shown that the results of the regulation of the traffic process at the studied real intersection, obtained by modeling using reinforcement learning, are superior to the optimization approach, but are sensitive to the given intensity schedule.