TCP-PPCC: Online-Learning Proximal Policy for Congestion Control

Effective network congestion control strategies are the key to secure the normal operation of complex and changeable networks. The fundamental assumptions of many existing TCP congestion control variants dominated by hand-crafted heuristic algorithms are no longer valid. We propose an algorithm called TCP-Proximal Policy Congestion Control (TCP-PPCC), which is based on deep reinforcement learning algorithm Proximal Policy Optimization (PPO). TCP-PPCC updates the policy offline from the features of the preceding network state and feedback from the current network environment and adjusts the congestion window online with the updated policy. The senders with TCP-PPCC can learn about the changes in network bandwidth more accurately and adjust the congestion window in time. We demonstrate the performance of TCP-PPCC by comparing it with the traditional congestion control algorithm NewReno in four network scenarios with the ns-3 simulator. The results show that in scenario 2, TCP-PPCC takes 58.75% improvement in average delay and 27.80% improvement in throughput compared with NewReno.