An Intelligent TCP Congestion Control Method Based on Deep Q Network

To optimize the data migration performance between different supercomputing centers in China, we present TCP-DQN, which is an intelligent TCP congestion control method based on DQN (Deep Q network). The TCP congestion control process is abstracted as a partially observed Markov decision process. In this process, an agent is constructed to interact with the network environment. The agent adjusts the size of the congestion window by observing the characteristics of the network state. The network environment feeds back the reward to the agent, and the agent tries to maximize the expected reward in an episode. We designed a weighted reward function to balance the throughput and delay. Compared with traditional Q-learning, DQN uses double-layer neural networks and experience replay to reduce the oscillation problem that may occur in gradient descent. We implemented the TCP-DQN method and compared it with mainstream congestion control algorithms such as cubic, Highspeed and NewReno. The results show that the throughput of TCP-DQN can reach more than 2 times of the comparison method while the latency is close to the three compared methods.

Improved RTT Fairness of BBR Congestion Control Algorithm Based on Adaptive Congestion Window

To alleviate the lower performance of Transmission Control Protocol (TCP) congestion control over complex network, especially the high latency and packet loss scenario, Google proposed the Bottleneck Bandwidth and Round-trip propagation time (BBR) congestion control algorithm. In contrast with other TCP congestion control algorithms, BBR adjusted transfer data by maximizing delivery rate and minimizing delay. However, some evaluation experiments have shown that the persistent queues formation and retransmissions in the bottleneck can lead to serious fairness issues between BBR flows with different round-trip times (RTTs). They pointed out that small RTT differences cause unfairness in the throughput of BBR flows and flows with longer RTT can obtain higher bandwidth when competing with the shorter RTT flows. In order to solve this fairness problem, an adaptive congestion window of BBR is proposed, which adjusts the congestion window gain of each BBR flow in network load. The proposed algorithms alleviate the RTT fairness issue by controlling the upper limit of congestion window according to the delivery rate and queue status. In the Network Simulator 3 (NS3) simulation experiment, it shows that the adaptive congestion window of BBR (BBR-ACW) congestion control algorithm improves the fairness by more than 50% and reduces the queuing delay by 54%, compared with that of the original BBR in different buffer sizes.