scholarly journals A Reinforcement Learning Approach for Interference Management in Heterogeneous Wireless Networks

2021 ◽  
Vol 15 (12) ◽  
pp. 65
Author(s):  
Akindele Segun Afolabi ◽  
Shehu Ahmed ◽  
Olubunmi Adewale Akinola
Keyword(s):  
Reinforcement Learning ◽  
Power Level ◽  
Heterogeneous Wireless Networks ◽  
Interference Management ◽  
Base Station ◽  
User Equipment ◽  
Base Stations ◽  
Multi Agent Systems ◽  
Q Learning ◽  
Macro Cell

<span lang="EN-US">Due to the increased demand for scarce wireless bandwidth, it has become insufficient to serve the network user equipment using macrocell base stations only. Network densification through the addition of low power nodes (picocell) to conventional high power nodes addresses the bandwidth dearth issue, but unfortunately introduces unwanted interference into the network which causes a reduction in throughput. This paper developed a reinforcement learning model that assisted in coordinating interference in a heterogeneous network comprising macro-cell and pico-cell base stations. The learning mechanism was derived based on Q-learning, which consisted of agent, state, action, and reward. The base station was modeled as the agent, while the state represented the condition of the user equipment in terms of Signal to Interference Plus Noise Ratio. The action was represented by the transmission power level and the reward was given in terms of throughput. Simulation results showed that the proposed Q-learning scheme improved the performances of average user equipment throughput in the network. In particular, </span><span lang="EN-US">multi-agent systems with a normal learning rate increased the throughput of associated user equipment by a whooping 212.5% compared to a macrocell-only scheme.</span>

scholarly journals Optimising Performance for NB-IoT UE Devices through Data Driven Models

2021 ◽  
Vol 10 (1) ◽  
pp. 21
Author(s):  
Omar Nassef ◽  
Toktam Mahmoodi ◽  
Foivos Michelinakis ◽  
Kashif Mahmood ◽  
Ahmed Elmokashfi
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Narrow Band ◽  
Learning Algorithm ◽  
Base Station ◽  
User Equipment ◽  
Data Driven ◽  
Superior Performance

This paper presents a data driven framework for performance optimisation of Narrow-Band IoT user equipment. The proposed framework is an edge micro-service that suggests one-time configurations to user equipment communicating with a base station. Suggested configurations are delivered from a Configuration Advocate, to improve energy consumption, delay, throughput or a combination of those metrics, depending on the user-end device and the application. Reinforcement learning utilising gradient descent and genetic algorithm is adopted synchronously with machine and deep learning algorithms to predict the environmental states and suggest an optimal configuration. The results highlight the adaptability of the Deep Neural Network in the prediction of intermediary environmental states, additionally the results present superior performance of the genetic reinforcement learning algorithm regarding its performance optimisation.

Improvement on Supporting Machine Learning Algorithm for Solving Problem in Immediate Decision Making

2012 ◽  
Vol 566 ◽  
pp. 572-579
Author(s):  
Abdolkarim Niazi ◽  
Norizah Redzuan ◽  
Raja Ishak Raja Hamzah ◽  
Sara Esfandiari
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Multi Agent Systems ◽  
Combined Model ◽  
Q Learning ◽  
Agent Systems ◽  
Multi Agent ◽  
Case Base ◽  
Case Base Reasoning ◽  
Robotic Tool

In this paper, a new algorithm based on case base reasoning and reinforcement learning (RL) is proposed to increase the convergence rate of the reinforcement learning algorithms. RL algorithms are very useful for solving wide variety decision problems when their models are not available and they must make decision correctly in every state of system, such as multi agent systems, artificial control systems, robotic, tool condition monitoring and etc. In the propose method, we investigate how making improved action selection in reinforcement learning (RL) algorithm. In the proposed method, the new combined model using case base reasoning systems and a new optimized function is proposed to select the action, which led to an increase in algorithms based on Q-learning. The algorithm mentioned was used for solving the problem of cooperative Markov’s games as one of the models of Markov based multi-agent systems. The results of experiments Indicated that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy.

scholarly journals Performance measurement of small-cells deployed under a heterogeneous network: an analysis of the co-existence of small-cells with the macro‑cell in 5G NR

2021 ◽  
Author(s):  
Mobasshir Mahbub ◽  
Bobby Barua
Keyword(s):  
Mobile Networks ◽  
Heterogeneous Network ◽  
Power Efficiency ◽  
Base Station ◽  
Small Cell ◽  
Base Stations ◽  
Small Cells ◽  
Macro Cell ◽  
Received Power ◽  
Implementation Costs

Abstract Advancements of cellular networks such as 4G and 5G proposed the collaboration of small-cell technologies in mobile networks and constructed a heterogeneous network (HetNet) for collaborative connectivity. There are many benefits of small-cell-based collective communication such as the increase of device capability in indoor/outdoor locations, enhancement of wireless coverage, improved signal efficiency, lower implementation costs of gNB (Next-generation Base Station introduced in 5G), etc. The integration of small-cells by deploying low-power BSs (base stations) in conventional macro-gNBs was investigated as a convenient and economical way of raising the potentials of a cellular network with high demand from consumers. The fusion of small-cells with macro-cells offers increased coverage and capacity for heterogeneous networks. Therefore, the research aimed to realize the performance of a small-cell deployed under a macro-cell in a two-tier heterogeneous network. The research first modified the reference equation for measuring the received power by introducing the transmitter and receiver gain. The paper then measured the SINR, throughput, spectral efficiency, and power efficiency for both downlink and uplink by empirical simulation. The research further enlisted the notable outcomes after examining the simulation results and discussed some relevant research scopes in the concluding sections of the paper.

scholarly journals Multiagent reinforcement learning using Non-Parametric Approximation

Respuestas ◽  
2018 ◽  
Vol 23 (2) ◽  
pp. 53-61
Author(s):  
David Luviano Cruz ◽  
Francesco José García Luna ◽  
Luis Asunción Pérez Domínguez
Keyword(s):  
Reinforcement Learning ◽  
Hybrid Control ◽  
Learning Algorithm ◽  
Multi Agent Systems ◽  
Generation Task ◽  
Q Learning ◽  
Agent Systems ◽  
Multi Agent ◽  
Optimal Set ◽  
Parametric Approximation

This paper presents a hybrid control proposal for multi-agent systems, where the advantages of the reinforcement learning and nonparametric functions are exploited. A modified version of the Q-learning algorithm is used which will provide data training for a Kernel, this approach will provide a sub optimal set of actions to be used by the agents. The proposed algorithm is experimentally tested in a path generation task in an unknown environment for mobile robots.

Channel Assignment Strategies for Mobile Networks Using Fuzzy Petri Nets

2011 ◽  
Vol 467-469 ◽  
pp. 1662-1667
Author(s):  
Yi Shun Weng ◽  
Yi Sheng Huang
Keyword(s):  
Petri Nets ◽  
Mobile Networks ◽  
Channel Assignment ◽  
Power Level ◽  
Base Station ◽  
Base Stations ◽  
Mobile Cellular ◽  
Mobile Cellular Networks ◽  
Fuzzy Petri Nets ◽  
Selection Of

In mobile cellular networks, the mobile devices need to handoff to different base stations based on certain criteria. And also fuzzy Petri nets can support an effective rule to deduce the inexact information. Based on the reasons, this paper focuses on the use of fuzzy Petri nets to model the handoff region for obtaining optimal channel assignment schemes. In this paper, a fuzzy logic based scheme for selection of base station is presented. The scheme considers two cover regions, namely, dual-BSs fuzzy assignment handoff and triple-BSs fuzzy assignment handoff of each base station to arrive at a fuzzy handoff decision regarding handoff to any particular base station. For comparison, the conventional power level based handoff scheme is also considered.

scholarly journals Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization

Electronics ◽  
2021 ◽  
Vol 10 (23) ◽  
pp. 2953
Author(s):  
Sudheesh Puthenveettil Gopi ◽  
Maurizio Magarini
Keyword(s):  
Reinforcement Learning ◽  
Clustering Algorithm ◽  
Base Station ◽  
Data Rate ◽  
Cellular System ◽  
Q Learning ◽  
Data Rates ◽  
In The Beginning ◽  
Exploratory Movements ◽  
Station Location

The application of unmanned aerial vehicles (UAV) as base station (BS) is gaining popularity. In this paper, we consider maximization of the overall data rate by intelligent deployment of UAV BS in the downlink of a cellular system. We investigate a reinforcement learning (RL)-aided approach to optimize the position of flying BSs mounted on board UAVs to support a macro BS (MBS). We propose an algorithm to avoid collision between multiple UAVs undergoing exploratory movements and to restrict UAV BSs movement within a predefined area. Q-learning technique is used to optimize UAV BS position, where the reward is equal to sum of user equipment (UE) data rates. We consider a framework where the UAV BSs carry out exploratory movements in the beginning and exploitary movements in later stages to maximize the overall data rate. Our results show that a cellular system with three UAV BSs and one MBS serving 72 UE reaches 69.2% of the best possible data rate, which is identified by brute force search. Finally, the RL algorithm is compared with a K-means algorithm to study the need of accurate UE locations. Our results show that the RL algorithm outperforms the K-means clustering algorithm when the measure of imperfection is higher. The proposed algorithm can be made use of by a practical MBS–UAV BSs–UEs system to provide protection to UAV BSs while maximizing data rate.

scholarly journals Location-Based Resource Allocation in Ultra-Dense Network with Clustering

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4022
Author(s):  
Seong Jung Kim ◽  
Jeong Gon Kim
Keyword(s):  
Resource Allocation ◽  
Communication Systems ◽  
Base Station ◽  
User Equipment ◽  
Base Stations ◽  
System Capacity ◽  
Subcarrier Allocation ◽  
Dense Network ◽  
Small Base ◽  
Ultra Dense Network

With the rapid deployment of present-day mobile communication systems, user traffic requirements have increased tremendously. An ultra-dense network is a configuration in which the density of small base stations is greater than or equal to that of the user equipment. Ultra-dense networks are considered as the key technology for 5th generation networks as they can improve the link quality and increase the system capacity. However, in an ultra-dense network, small base stations are densely positioned, so one user equipment may receive signals from two or more small base stations. This may cause a severe inter-cell interference problem. In this study, we considered a coordinated multi-point scenario, a cooperative technology between base stations to alleviate the interference. In addition, to suppress the occurrence of severe interference at the cell edges, link formation was carried out by considering the degree of cell load for each cluster. After the formation of links between all the base stations and user equipment, a subcarrier allocation procedure was performed. The subcarrier allocation method used in this study was based on the location of base stations with clustering to improve the data rate and reduce the interference between the clusters. Power allocation was based on the channel gain between the base station and user equipment. Simulation results showed that the proposed scheme delivered a higher sum rate than the other resource allocation methods reported previously for various types of user equipment.

scholarly journals Optimizing Macro and Micro Base Stations Locations and User Associations in Heterogeneous Wireless Networks

2021 ◽  
Author(s):  
Noha Hassan
Keyword(s):  
Real Time ◽  
Coverage Probability ◽  
Communication Systems ◽  
Dimensional Space ◽  
Heterogeneous Wireless Networks ◽  
Base Station ◽  
Base Stations ◽  
Sequence Length ◽  
User Association ◽  
Signal Power

Heterogeneous Networks (HetNets) have gained the attraction of the communication industry recently, due to their promising ability to enhance the performance of future broadband Fifth Generation (5G) networks and are integral parts of 5G systems. They can be viewed in multi-dimensional space where, each slice represents a unique tier that has its own Base Station (BS)s and User Equipment (UE)s. Different tiers cooperate with each other for their mutual benefit. Data can be interactively exchanged among the tiers, and UEs have the flexibility to switch between the tiers. The cells in such a heterogeneous cellular networks have variable sizes, shapes, and coverage regions. However, in HetNets with ultra dense BSs, the distance between them gets very small and, they suffer from very high levels of mutual interference. To improve the performance of HetNets, we have done multiple contributions in this dissertation. First, we have developed analytical derivations for optimizing pilot sequence length which is a very crucial factor in acquiring the Channel State Information (CSI) and the channel estimation process in general. Poisson Point Process (PPP) has been widely used to allocate BSs among various tiers so far. However, BS locations obtained using PPP approach may not be optimum to reduce interference. Therefore, in this dissertation, BSs locations are optimized to reduce the interference and improve the coverage and received signal power. Also, we have derived expressions for static UEs coverage probability and network energy efficiency in HetNets. A proper UE association algorithm for HetNets is a great challenge. The classic max-Signal to Interference and Noise Ratio (SINR) or max-received signal strength (RSS) user association algorithms are inappropriate solutions for HetNets as UEs in this context will tend to connect to the Macro BS, which is the one with the highest signal power. A severe load imbalance and significant inefficiency arises and impacts the performance. The aforementioned algorithms tend to associate UEs to BSs with the best received signal power or signal quality. In HetNets, usually Macro BSs are the ones transmitting the strongest signals; hence most UEs tend to associate with the Macro BS leaving Micro BSs with less load. Also, the conventional max-SINR and max-RSS algorithms do not provide adequate results in multi-tier systems. We suggest two centralized algorithms, LSTD and RTLB, for an even UE association to provide fair load distribution. However RTLB outperforms LSTD in real time scenarios as it easily and quickly adapts to rapid network changes. Furthermore, we consider the mobility of nodes. We derive coverage probability for moving UEs considering both handover and no handover scenarios. Proposed algorithms are fast enough to associate the moving users to different Micro and Macro BSs appropriately in real time. Our algorithms are proved to be feasible and provide a path towards attainable future communication systems.

scholarly journals Reinforcement Learning Based Hierarchical Multi-Agent Robotic Search Team in Uncertain Environment

2021 ◽  
Vol 40 (3) ◽  
pp. 645-662
Author(s):  
Shahzaib Hamid ◽  
Ali Nasir ◽  
Yasir Saleem
Keyword(s):  
Reinforcement Learning ◽  
Multi Agent Systems ◽  
Qualitative Comparison ◽  
Q Learning ◽  
Novel Approach ◽  
Learning Agent ◽  
Markov Decision ◽  
Multi Agent ◽  
Efficient Learning ◽  
Prior Models

Field of robotics has been under the limelight because of recent advances in Artificial Intelligence (AI). Due to increased diversity in multi-agent systems, new models are being developed to handle complexity of such systems. However, most of these models do not address problems such as; uncertainty handling, efficient learning, agent coordination and fault detection. This paper presents a novel approach of implementing Reinforcement Learning (RL) on hierarchical robotic search teams. The proposed algorithm handles uncertainties in the system by implementing Q-learning and depicts enhanced efficiency as well as better time consumption compared to prior models. The reason for that is each agent can take action on its own thus there is less dependency on leader agent for RL policy. The performance of this algorithm is measured by introducing agents in an unknown environment with both Markov Decision Process (MDP) and RL policies at their disposal. Simulation-based comparison of the agent motion is presented using the results from of MDP and RL policies. Furthermore, qualitative comparison of the proposed model with prior models is also presented.

scholarly journals Intelligent Interference Management in UAV-Based HetNets

Telecom ◽  
2021 ◽  
Vol 2 (4) ◽  
pp. 472-488
Author(s):  
Simran Singh ◽  
Abhaykumar Kumbhar ◽  
İsmail Güvenç ◽  
Mihail L. Sichitiu
Keyword(s):  
Cellular Networks ◽  
Hot Spots ◽  
Network Performance ◽  
Interference Management ◽  
Base Stations ◽  
Computationally Efficient ◽  
Q Learning ◽  
Conventional Optimization ◽  
Coverage Holes ◽  
Mobile Base

Unmanned aerial vehicles (UAVs) can play a key role in meeting certain demands of cellular networks. UAVs can be used not only as user equipment (UE) in cellular networks but also as mobile base stations (BSs) wherein they can either augment conventional BSs by adapting their position to serve the changing traffic and connectivity demands or temporarily replace BSs that are damaged due to natural disasters. The flexibility of UAVs allows them to provide coverage to UEs in hot-spots, at cell-edges, in coverage holes, or regions with scarce cellular infrastructure. In this work, we study how UAV locations and other cellular parameters may be optimized in such scenarios to maximize the spectral efficiency (SE) of the network. We compare the performance of machine learning (ML) techniques with conventional optimization approaches. We found that, on an average, a double deep Q learning approach can achieve 93.46% of the optimal median SE and 95.83% of the optimal mean SE. A simple greedy approach, which tunes the parameters of each BS and UAV independently, performed very well in all the cases that we tested. These computationally efficient approaches can be utilized to enhance the network performance in existing cellular networks.

Sign in / Sign up

Export Citation Format

Share Document