Cognitive mesh network resource adaptations using reinforcement learning

Cognitive radio networks (CRNs) can provide a means for offering end-to-end Quality of Service (QoS) required by unlicensed users (secondary users. SUs). The authors consider the approach where licensed users (primary users, PUs) play the role of routers and lease spectrum with QoS guarantees for the SUs. Available spectrum is managed by the PU admission and routing policy. The main concern of the proposed policy is to provide end-to-end QoS connections to the SUs. Maximizing gain is the key objective for the PU. In this paper, the authors propose a novel resource management scheme where reinforcement learning (RL) is used to drive resource management scheme. The derived scheme helps PUs to adapt to the changes in the network conditions such as traffic load, spectrum cost, service reward, etc, so that PU's gain can continuously be optimized. The approach integrates spectrum adaptations with connection admission control and routing policies. Numerical analysis results show the ability of the proposed approach to attain the optimal gain under different conditions and constraints.

Download Full-text

A STUDY OF ROUTING METHODS IN URBAN WIRELESS MESH NETWORKS BASED ON REINFORCEMENT LEARNING

Труды НИИР ◽

10.34832/niir.2021.5.2.001 ◽

2021 ◽

Author(s):

В.Д. ФАМ ◽

Р.В. КИРИЧЕК ◽

А.С. БОРОДИН

Keyword(s):

Artificial Intelligence ◽

Reinforcement Learning ◽

Simulation Model ◽

Network Traffic ◽

Mesh Networks ◽

Mesh Network ◽

Queueing Model ◽

Traffic Routing ◽

Wireless Mesh ◽

Routing Methods

Приведены результаты исследования методов маршрутизации на основе обучения с подкреплением с помощью имитационной модели. Рассмотрена задача маршрутизации сетевого трафика для фрагмента ячеистой сети городского масштаба, управляемой на основе технологий искусственного интеллекта. Представлена модель системы массового обслуживания для изучения процесса маршрутизации, а также обучения выбора маршрута. Имитационная модель фрагмента ячеистой сети разработана в пакете Anylogic и обучается на основе платформы Microsoft Bonsai. The results of the study of network traffic routing methods based on reinforcement learning using a simulation model are presented. The problem of network traffic routing for a fragment of a city-scale mesh network, controlled on the basis of artificial intelligence technologies, is considered. The article presents a queueing model for studying the routing process, as well as learning how to choose a route. The mesh network fragment simulation model was developed in the Anylogic package and is trained on the basis of the Microsoft Bonsai platform.

Download Full-text

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

Applied Sciences ◽

10.3390/app112210870 ◽

2021 ◽

Vol 11 (22) ◽

pp. 10870

Author(s):

Abdikarim Mohamed Ibrahim ◽

Kok-Lim Alvin Yau ◽

Yung-Wey Chong ◽

Celimuge Wu

Keyword(s):

Resource Allocation ◽

Reinforcement Learning ◽

Network Routing ◽

Network Resource ◽

Network Resource Allocation ◽

Significant Performance ◽

Multi Agent ◽

Taxonomic Approach ◽

And Performance ◽

Reinforcement Learning Models

Recent advancements in deep reinforcement learning (DRL) have led to its application in multi-agent scenarios to solve complex real-world problems, such as network resource allocation and sharing, network routing, and traffic signal controls. Multi-agent DRL (MADRL) enables multiple agents to interact with each other and with their operating environment, and learn without the need for external critics (or teachers), thereby solving complex problems. Significant performance enhancements brought about by the use of MADRL have been reported in multi-agent domains; for instance, it has been shown to provide higher quality of service (QoS) in network resource allocation and sharing. This paper presents a survey of MADRL models that have been proposed for various kinds of multi-agent domains, in a taxonomic approach that highlights various aspects of MADRL models and applications, including objectives, characteristics, challenges, applications, and performance measures. Furthermore, we present open issues and future directions of MADRL.

Download Full-text

Actor-Critic Wavelet Neural Network based Scheduler Technique for LTE-Advanced

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.d7673.018520 ◽

2020 ◽

Vol 8 (5) ◽

pp. 4856-4863

Keyword(s):

Neural Network ◽

Reinforcement Learning ◽

Large Scale ◽

Simulation Analysis ◽

Resource Scheduling ◽

Wavelet Neural Network ◽

Time Interval ◽

Sigmoid Function ◽

Network Resource ◽

Actor Network

This work presents an efficient and intelligent resource scheduling strategy for the Long Term EvolutionAdvanced (LTE-A) downlink transmission using Reinforcement learning and wavelet neural network. Resource scheduling in LTE-A suffers the problem of uncertainty and accuracy for large scale network. Also the performance of scheduling in conventional methods solely depends upon the scheduling algorithm which was fixed for the entire transmission session. This issue has been addressed and resolved in this paper through Actor-Critic architecture based reinforcement learning to provide the best suited scheduling method out of the rule set for every transmission time interval (TTI) of communication. The actor network will take the decision on scheduling and the critic network will evaluate this decision and update the actor network adaptively through the optimal tuning laws so as to get the desired performance in scheduling. Wavelet neural network(WNN) is derived here by using wavelet function as activation function in place of sigmoid function in conventional neural network to attain better learning capabilities, faster convergence and efficient decision making in scheduling. The actor and critic networks are created through these WNNs and are trained with the LTE parameters dataset. The efficacy of the presented work is evaluated through simulation analysis.

Download Full-text

Accelerating Deep Neuroevolution on Distributed FPGAs for Reinforcement Learning Problems

ACM Journal on Emerging Technologies in Computing Systems ◽

10.1145/3425500 ◽

2021 ◽

Vol 17 (2) ◽

pp. 1-17

Author(s):

Alexis Asseman ◽

Nicolas Antoine ◽

Ahmet S. Ozcan

Keyword(s):

Reinforcement Learning ◽

High Performance ◽

Mesh Network ◽

System Level ◽

Evolutionary Strategies ◽

Robotic Control ◽

Training Time ◽

The Neural Network ◽

Representational Power ◽

Level Parallelism

Reinforcement learning, augmented by the representational power of deep neural networks, has shown promising results on high-dimensional problems, such as game playing and robotic control. However, the sequential nature of these problems poses a fundamental challenge for computational efficiency. Recently, alternative approaches such as evolutionary strategies and deep neuroevolution demonstrated competitive results with faster training time on distributed CPU cores. Here we report record training times (running at about 1 million frames per second) for Atari 2600 games using deep neuroevolution implemented on distributed FPGAs. Combined hardware implementation of the game console, image preprocessing and the neural network in an optimized pipeline, multiplied with the system level parallelism enabled the acceleration. These results are the first application demonstration on the IBM Neural Computer, which is a custom designed system that consists of 432 Xilinx FPGAs interconnected in a 3D mesh network topology. In addition to high performance, experiments also showed improvement in accuracy for all games compared to the CPU implementation of the same algorithm.

Download Full-text

Network resource brokerage by means of distributed agent-based systems encompassing reinforcement learning schemes

Computer Communications ◽

10.1016/s0140-3664(02)00043-9 ◽

2002 ◽

Vol 25 (16) ◽

pp. 1415-1428 ◽

Cited By ~ 2

Author(s):

M. Saltouros ◽

A. Taskaris ◽

P. Demestichas ◽

M. Theologou ◽

A. Vasilakos

Keyword(s):

Reinforcement Learning ◽

Network Resource ◽

Agent Based ◽

Learning Schemes

Download Full-text

Network resource optimization with reinforcement learning for low power wide area networks

EURASIP Journal on Wireless Communications and Networking ◽

10.1186/s13638-020-01783-5 ◽

2020 ◽

Vol 2020 (1) ◽

Author(s):

Gyubong Park ◽

Wooyeob Lee ◽

Inwhee Joe

Keyword(s):

Reinforcement Learning ◽

Wireless Communication ◽

Low Power ◽

Industrial Revolution ◽

Data Rate ◽

Wide Area ◽

Transmission Power ◽

Long Distance ◽

Network Resource ◽

Transmission Parameters

Abstract As the 4th industrial revolution using information becomes an issue, wireless communication technologies such as the Internet of Things have been spotlighted. Therefore, much research is needed to satisfy the technological demands for the future society. A LPWA (low power wide area) in the wireless communication environment enables low-power, long-distance communication to meet various application requirements that conventional wireless communications have been difficult to meet. We propose a method to consume the minimum transmission power relative to the maximum data rate with the target of LoRaWAN among LPWA networks. Reinforcement learning is adopted to find the appropriate parameter values for the minimum transmission power. With deep reinforcement learning, we address the LoRaWAN problem with the goal of optimizing the distribution of network resources such as spreading factor, transmission power, and channel. By creating a number of deep reinforcement learning agents that match the terminal nodes in the network server, the optimal transmission parameters are provided to the terminal nodes. The simulation results show that the proposed method is about 15% better than the existing ADR (adaptive data rate) MAX of LoRaWAN in terms of throughput relative to energy transmission.

Download Full-text