PPORM: A PPO-assisted packet reordering mechanism of heterogeneous VANETs for enhancing goodput and stability in fog computing

Integrated Vehicular Networks (VANETs) constructed through the collaboration of various heterogeneous networks, such as 4G, 5G, satellite networks, and Unmanned Aerial Vehicle (UAV) networks, provide an effective solution to the resource constraints between vehicles and edge fog computing nodes. Reordering Buffer (RB) is crucial in concurrent data transmission between vehicles and edge fog computing nodes via heterogeneous VANETs. RB is in charge of storing out-of-order packets, waiting for packets with smaller sequence numbers, and delivering in-order packets to upper-layer applications. However, current packet reordering mechanisms are challenging in providing stable and high goodput due to the inappropriate timeout timers and uneven delivery rules. In this paper, we propose a PPO-assisted packet reordering mechanism (PPORM) to achieve optimal control of packet delivery. We first transform the goodput maximization problem into the optimal timeout threshold of RB and the optimal delivery moment of each packet. Secondly, we introduce a Proximal Policy Optimization-assisted Timeout Threshold Updating (TTU) algorithm to dynamically adjust the threshold in response to real-time changes in network conditions. Further, we present a Multifactor Smooth Delivery (MSD) algorithm to regulate the optimal queuing delay for each packet and enhance the stability of the real-time throughput as much as possible. Experimental results show that PPORM improves goodput by $6.94\text{\%}\sim 45.57\text{\%}$ and improves stability by $32.5\text{\%}\sim 49.58\text{\%}$ compared with other baseline algorithms.