Towards wireless time-sensitive networking: Multi-link deterministic scheduling via deep reinforcement learning

Abstract

Wireless time-sensitive networking (WTSN) has gained significant popularity because of its potential flexibility, scalability and bounded latency. Nevertheless, fewer studies have investigated specific scheduling strategies under unreliable wireless media. Besides, the lack of interoperability with current industrial networks and the ability to handle complex wireless network scheduling pose great challenges in WTSN design. To handle these challenges, we first propose a WiFi-based WTSN architecture. The wireless time-aware shaper (WTAS) and deep reinforcement learning based flow scheduling strategy are studied as the core mechanism of the proposed architecture. Specifically, the functionality of WTSN is designed through wireless gate control, control-aware retransmission, deterministic time slot design and delay analysis, and dynamic queue priority mapping. Then, we formulate a multi-link time slot scheduling problem to achieve load balance while considering transmission and control constraints. The soft actor-critic-based flow scheduling optimization (SAC-FSO) algorithm with elaborately designed decision variables mapping and dynamic constraint detection is proposed to efficiently achieve the wireless deterministic transmission. The performance evaluation demonstrates the effectiveness of the proposed WTSN architecture and WTAS modules. Comparison results with other algorithms demonstrate that the SAC-FSO algorithm is more efficient and scalable.