Multi-Agent Q-Learning Based Multi-UAV Wireless Networks for Maximizing Energy Efficiency: Deployment and Power Control Strategy Design

Author(s):  
Seungmin Lee ◽  
Heejung Yu ◽  
Howon Lee
Energies ◽  
2019 ◽  
Vol 12 (22) ◽  
pp. 4300 ◽  
Author(s):  
Hoon Lee ◽  
Han Seung Jang ◽  
Bang Chul Jung

Achieving energy efficiency (EE) fairness among heterogeneous mobile devices will become a crucial issue in future wireless networks. This paper investigates a deep learning (DL) approach for improving EE fairness performance in interference channels (IFCs) where multiple transmitters simultaneously convey data to their corresponding receivers. To improve the EE fairness, we aim to maximize the minimum EE among multiple transmitter–receiver pairs by optimizing the transmit power levels. Due to fractional and max-min formulation, the problem is shown to be non-convex, and, thus, it is difficult to identify the optimal power control policy. Although the EE fairness maximization problem has been recently addressed by the successive convex approximation framework, it requires intensive computations for iterative optimizations and suffers from the sub-optimality incurred by the non-convexity. To tackle these issues, we propose a deep neural network (DNN) where the procedure of optimal solution calculation, which is unknown in general, is accurately approximated by well-designed DNNs. The target of the DNN is to yield an efficient power control solution for the EE fairness maximization problem by accepting the channel state information as an input feature. An unsupervised training algorithm is presented where the DNN learns an effective mapping from the channel to the EE maximizing power control strategy by itself. Numerical results demonstrate that the proposed DNN-based power control method performs better than a conventional optimization approach with much-reduced execution time. This work opens a new possibility of using DL as an alternative optimization tool for the EE maximizing design of the next-generation wireless networks.


Author(s):  
Yaojin Xu ◽  
Long Di ◽  
YangQuan Chen

Small unmanned aerial vehicles (UAVs) can provide facilities in various applications. Compared with single UAV system, small UAVs based cooperative UAV system can bring advantages such as higher efficiency and safety. Therefore, it is necessary to design a robust multi-agent cooperative flight controller to coordinate a group of small UAVs for stable formation flights. This paper investigates the problem of consensus-based formation control for a multi-UAV system. Firstly, We choose a simplified model with nonholonomic constraints for UAV dynamics. Secondly, using the algebraic theory and backstepping technique, we design consensus protocols for multi-UAV systems under the strongly connected topology. Then based on that, we propose a multi-UAVfromation control strategy. Finally, we extend our results to the directed topology case which is still effective by simulation.


Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2361 ◽  
Author(s):  
Hyebin Park ◽  
Yujin Lim

Increased data traffic resulting from the increase in the deployment of connected vehicles has become relevant in vehicular social networks (VSNs). To provide efficient communication between connected vehicles, researchers have studied device-to-device (D2D) communication. D2D communication not only reduces the energy consumption and loads of the system but also increases the system capacity by reusing cellular resources. However, D2D communication is highly affected by interference and therefore requires interference-management techniques, such as mode selection and power control. To make an optimal mode selection and power control, it is necessary to apply reinforcement learning that considers a variety of factors. In this paper, we propose a reinforcement-learning technique for energy optimization with fifth-generation communication in VSNs. To achieve energy optimization, we use centralized Q-learning in the system and distributed Q-learning in the vehicles. The proposed algorithm learns to maximize the energy efficiency of the system by adjusting the minimum signal-to-interference plus noise ratio to guarantee the outage probability. Simulations were performed to compare the performance of the proposed algorithm with that of the existing mode-selection and power-control algorithms. The proposed algorithm performed the best in terms of system energy efficiency and achievable data rate.


2014 ◽  
Vol 556-562 ◽  
pp. 1766-1769 ◽  
Author(s):  
Lian Fen Huang ◽  
Bin Wen ◽  
Zhi Bin Gao ◽  
Hong Xiang Cai ◽  
Yu Jie Li

Femtocell is introduced to improve indoor coverage, which is beneficial for both users and operators. But it will also inevitably produce interference management issues in the heterogeneous network which consists of femtocells and macrocells. In this paper, a decentralized Q-learning-based power control strategy is proposed, comparing with homogenous power allocation and smart power control (SPC) algorithm. Simulation results have shown that Q-learning-based power control algorithm can implement the compromise of capacity between macrocells and femtocells, and greatly enhance energy efficiency of the whole network.


2020 ◽  
Vol 73 (4) ◽  
pp. 874-891
Author(s):  
Wenjie Zhao ◽  
Zhou Fang ◽  
Zuqiang Yang

A distributed four-dimensional (4D) trajectory generation method based on multi-agent Q learning is presented for multiple unmanned aerial vehicles (UAVs). Based on this method, each vehicle can intelligently generate collision-free 4D trajectories for time-constrained cooperative flight tasks. For a single UAV, the 4D trajectory is generated by the bionic improved tau gravity guidance strategy, which can synchronously guide the position and velocity to the desired values at the arrival time. Furthermore, to optimise trajectory parameters, the continuous state and action wire fitting neural network Q (WFNNQ) learning method is applied. For multi-UAV applications, the learning is organised by the win or learn fast-policy hill climbing (WoLF-PHC) algorithm. Dynamic simulation results show that the proposed method can efficiently provide 4D trajectories for the multi-UAV system in challenging simultaneous arrival tasks, and the fully trained method can be used in similar trajectory generation scenarios.


Sign in / Sign up

Export Citation Format

Share Document