Double Deep Recurrent Reinforcement Learning for Centralized Dynamic Multichannel Access

We consider the problem of dynamic multichannel access for transmission maximization in multiuser wireless communication networks. The objective is to find a multiuser strategy that maximizes global channel utilization with a low collision in a centralized manner without any prior knowledge. Obtaining an optimal solution for centralized dynamic multichannel access is an extremely difficult problem due to the large-state and large-action space. To tackle this problem, we develop a centralized dynamic multichannel access framework based on double deep recurrent Q-network. The centralized node first maps current state directly to channel assignment actions, which can overcome prohibitive computation compared with reinforcement learning. Then, the centralized node can be easy to select multiple channels by maximizing the sum of value functions based on a trained neural network. Finally, the proposed method avoids collisions between secondary users through centralized allocation policy.

Download Full-text

Multi-Agent Actor Critic for Channel Allocation in Heterogeneous Networks

International Journal of Mobile Computing and Multimedia Communications ◽

10.4018/ijmcmc.2020010102 ◽

2020 ◽

Vol 11 (1) ◽

pp. 23-41

Author(s):

Nan Zhao ◽

Zehua Liu ◽

Yiqiang Cheng ◽

Chao Tian

Keyword(s):

Reinforcement Learning ◽

Communication Networks ◽

Heterogeneous Networks ◽

High Efficiency ◽

Channel Allocation ◽

Optimal Solution ◽

New Method ◽

Multi Agent ◽

Optimal Approach ◽

The Cost

Heterogeneous networks (HetNets) can equalize traffic loads and cut down the cost of deploying cells. Thus, it is regarded to be the significant technique of the next-generation communication networks. Due to the non-convexity nature of the channel allocation problem in HetNets, it is difficult to design an optimal approach for allocating channels. To ensure the user quality of service as well as the long-term total network utility, this article proposes a new method through utilizing multi-agent reinforcement learning. Moreover, for the purpose of solving computational complexity problem caused by the large action space, deep reinforcement learning is put forward to learn optimal policy. A nearly-optimal solution with high efficiency and rapid convergence speed could be obtained by this learning method. Simulation results reveal that this new method has the best performance than other methods.

Download Full-text

UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning

International Journal of Aerospace Engineering ◽

10.1155/2021/3360116 ◽

2021 ◽

Vol 2021 ◽

pp. 1-12

Author(s):

Baolai Wang ◽

Shengang Li ◽

Xianzhong Gao ◽

Tao Xie

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

State Space ◽

Optimal Solution ◽

Difficult Problem ◽

Action Space ◽

Sequential Decision ◽

Autonomous Decision ◽

Multiagent Reinforcement Learning ◽

Uav Swarm

With the development of unmanned aerial vehicle (UAV) technology, UAV swarm confrontation has attracted many researchers’ attention. However, the situation faced by the UAV swarm has substantial uncertainty and dynamic variability. The state space and action space increase exponentially with the number of UAVs, so that autonomous decision-making becomes a difficult problem in the confrontation environment. In this paper, a multiagent reinforcement learning method with macro action and human expertise is proposed for autonomous decision-making of UAVs. In the proposed approach, UAV swarm is modeled as a large multiagent system (MAS) with an individual UAV as an agent, and the sequential decision-making problem in swarm confrontation is modeled as a Markov decision process. Agents in the proposed method are trained based on the macro actions, where sparse and delayed rewards, large state space, and action space are effectively overcome. The key to the success of this method is the generation of the macro actions that allow the high-level policy to find a near-optimal solution. In this paper, we further leverage human expertise to design a set of good macro actions. Extensive empirical experiments in our constructed swarm confrontation environment show that our method performs better than the other algorithms.

Download Full-text

Deep Reinforcement Learning for Mobile Video Offloading in Heterogeneous Cellular Networks

International Journal of Mobile Computing and Multimedia Communications ◽

10.4018/ijmcmc.2018100103 ◽

2018 ◽

Vol 9 (4) ◽

pp. 34-57 ◽

Cited By ~ 2

Author(s):

Nan Zhao ◽

Chao Tian ◽

Menglin Fan ◽

Minghu Wu ◽

Xiao He ◽

...

Keyword(s):

Reinforcement Learning ◽

Cellular Networks ◽

Communication Networks ◽

Optimal Solution ◽

Mobile Video ◽

Heterogeneous Cellular Networks ◽

Optimal Method ◽

Q Learning ◽

Mobile Offloading

Heterogeneous cellular networks can balance mobile video loads and reduce cell arrangement costs, which is an important technology of future mobile video communication networks. Because of the characteristics of non-convexity of the mobile offloading problem, the design of the optimal strategy is an essential issue. For the sake of ensuring users' quality of service and the long-term overall network utility, this article proposes the distributive optimal method by means of multiple agent reinforcement learning in the downlink heterogeneous cellular networks. In addition, to solve the computational load issue generated by the large action space, deep reinforcement learning is introduced to gain the optimal policy. The learning policy can provide a near-optimal solution efficiently with a fast convergence speed. Simulation results show that the proposed approach is more efficient at improving the performance than the Q-learning method.

Download Full-text