Power Allocation and Energy Cooperation for UAV-Enabled MmWave Networks: A Multi-Agent Deep Reinforcement Learning Approach

Unmanned Aerial Vehicle (UAV)-assisted cellular networks over the millimeter-wave (mmWave) frequency band can meet the requirements of a high data rate and flexible coverage in next-generation communication networks. However, higher propagation loss and the use of a large number of antennas in mmWave networks give rise to high energy consumption and UAVs are constrained by their low-capacity onboard battery. Energy harvesting (EH) is a viable solution to reduce the energy cost of UAV-enabled mmWave networks. However, the random nature of renewable energy makes it challenging to maintain robust connectivity in UAV-assisted terrestrial cellular networks. Energy cooperation allows UAVs to send their excessive energy to other UAVs with reduced energy. In this paper, we propose a power allocation algorithm based on energy harvesting and energy cooperation to maximize the throughput of a UAV-assisted mmWave cellular network. Since there is channel-state uncertainty and the amount of harvested energy can be treated as a stochastic process, we propose an optimal multi-agent deep reinforcement learning algorithm (DRL) named Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to solve the renewable energy resource allocation problem for throughput maximization. The simulation results show that the proposed algorithm outperforms the Random Power (RP), Maximal Power (MP) and value-based Deep Q-Learning (DQL) algorithms in terms of network throughput.

Multi-Objective Optimization of Energy Saving and Throughput in Heterogeneous Networks Using Deep Reinforcement Learning

Wireless networking using GHz or THz spectra has encouraged mobile service providers to deploy small cells to improve link quality and cell capacity using mmWave backhaul links. As green networking for less CO2 emission is mandatory to confront global climate change, we need energy efficient network management for such denser small-cell heterogeneous networks (HetNets) that already suffer from observable power consumption. We establish a dual-objective optimization model that minimizes energy consumption by switching off unused small cells while maximizing user throughput, which is a mixed integer linear problem (MILP). Recently, the deep reinforcement learning (DRL) algorithm has been applied to many NP-hard problems of the wireless networking field, such as radio resource allocation, association and power saving, which can induce a near-optimal solution with fast inference time as an online solution. In this paper, we investigate the feasibility of the DRL algorithm for a dual-objective problem, energy efficient routing and throughput maximization, which has not been explored before. We propose a proximal policy (PPO)-based multi-objective algorithm using the actor-critic model that is realized as an optimistic linear support framework in which the PPO algorithm searches for feasible solutions iteratively. Experimental results show that our algorithm can achieve throughput and energy savings comparable to the CPLEX.

Secure Rate Control and Statistical QoS Provisioning for Cloud-Based IoT Networks

The Internet of Things (IoT) facilitates physical things to detect, interact, and execute activities on-demand, enabling a variety of applications such as smart homes and smart cities. However, it also creates many potential risks related to data security and privacy vulnerabilities on the physical layer of cloud-based Internet of Things (IoT) networks. These can include different types of physical attacks such as interference, eavesdropping, and jamming. As a result, quality-of-service (QoS) provisioning gets difficult for cloud-based IoT. This paper investigates the statistical QoS provisioning of a four-node cloud-based IoT network under security, reliability, and latency constraints by relying on the effective capacity model to offer enhanced QoS for IoT networks. Alice and Bob are legitimate nodes trying to communicate with secrecy in the considered scenario, while an eavesdropper Eve overhears their communication. Meanwhile, a friendly jammer, which emits artificial noise, is used to degrade the wiretap channel. By taking advantage of their multiple antennas, Alice implements transmit antenna selection, while Bob and Eve perform maximum-ratio combining. We further assume that Bob decodes the artificial noise perfectly and thus removes its contribution by implementing perfect successive interference cancellation. A closed-form expression for an alternative formulation of the outage probability, conditioned upon the successful transmission of a message, is obtained by considering adaptive rate allocation in an ON-OFF transmission. The data arriving at Alice’s buffer are modeled by considering four different Markov sources to describe different IoT traffic patterns. Then, the problem of secure throughput maximization is addressed through particle swarm optimization by considering the security, latency, and reliability constraints. Our results evidence the considerable improvements on the delay violation probability by increasing the number of antennas at Bob under strict buffer constraints.

Implementation of Equilibrium Strategy Aiming at Throughput Maximization of Series Battery Pack

In the operation process of a power battery pack, the inconsistency among lithium-ion cells may seriously restrict the pack’s capacity, power capability and lifetime, which may bring hidden danger to the use of electric vehicles. Equalization management systems (EMSs) are crucial to alleviate such inter-cell inconsistency, whose performance such as accuracy and stability, mainly depends on the setting of equalization control strategies. This paper proposes an equalization strategy aimed at throughput maximization of series battery in the whole life cycle based on Model Prediction Control (MPC). In this paper, a Mean-plus-difference model (M+D) model is selected as the series battery model and the parameters are identified by Recursive Least Squares (RLS). Based on the model predictive control theory, the control model of series battery pack is established and the objective function of maximizing the throughput in the whole life cycle is derived. At the end of the paper, the simulation results show that the proposed equalization strategy can achieve greater life cycle throughput compared with the traditional SOC equalization strategy, which verifies the guiding significance of the equilibrium strategy proposed in this paper.