Resource Allocation For D2D Communications With A Novel Distributed Q-Learning Algorithm In Heterogeneous Networks

Device-to-device (D2D) communications and femtocell systems can bring significant benefits to users’ throughput. However, the complicated three-tier interference among macrocell, femtocell, and D2D systems is a challenging issue in heterogeneous networks. As D2D user equipment (UE) can cause interference to cellular UE, scheduling and allocation of channel resources and power of D2D communication need elaborate coordination. In this paper, we propose a joint scheduling and resource allocation scheme to improve the performance of D2D communication. We take UE rate and UE fairness into account by performing interference management. First, we construct a Stackelberg game framework in which we group a macrocellular UE, a femtocellular UE, and a D2D UE to form a two-leader one-follower pair. The cellular UE are leaders, and D2D UE is the follower who buys channel resources from the leaders. We analyze the equilibrium of the game and obtain solutions to the equilibrium. Second, we propose an algorithm for joint scheduling of D2D pairs based on their utility. Finally, we perform computer simulations to study the performance of the proposed scheme.

Download Full-text

Resource allocation in clustered M2M networks: a q-learning approach

10.32920/ryerson.14649174.v1 ◽

2021 ◽

Author(s):

Fatima Hussain

Keyword(s):

Resource Allocation ◽

Learning Algorithm ◽

Random Access ◽

Cumulative Distribution ◽

Convergence Time ◽

Mixed Integer ◽

M2m Communication ◽

Q Learning ◽

Threshold Levels ◽

Slot Assignment

Machine to machine (M2M) communication has received increasing attention in recent years. A M2M network exhibits salient features such as large number of machines/devices, low data rates, delay tolerant/sensitive, small sized packets, energy-constrained and low or no mobility. A large number of M2M terminals may exist in a small area with many trying to simultaneously and randomly access for channel resources - which will result in overload and access problem. This increased signaling overhead and diverse requirements of machine type communication devices (MTCDs) call for the development of flexible and efficient scheduling and random access techniques. In this thesis, we first review and compare various scheduling and random access techniques in LTE-based cellular networks for M2M communication. We also discuss how successful they are to fulfill the unique requirements of M2M communication and networking. Resource management in M2M networks with a large number devices is also reviewed from the access point of view. We propose a multi-objective optimization based solution to the problem of resource allocation in interference-limited M2M communication. We consider MTCDs in a clustered network structure, where they are divided into clusters and the devices belonging to a cluster communicate to cluster head (or controller). We maximize the number of admitted MTCD controllers and throughput with least interference caused to conventional primary users. We formulate the problem as a mixed-integer non-linear problem with multiple objectives and solve it using meshed adaptive direct search (MADS) algorithm. Simulation results show the effects of varying different parameters on cumulative throughput and the number of admitted iii MTCD controllers. We then formulate the slot selection problem in M2M networks with admitted MTCDs as an optimization problem. We present a solution using the Q-learning algorithm to select conflict-free slot assignment in a random access network with MTCD controllers. The performance of the solution is dependent on parameters such as learning rate and reward. We thoroughly analyze the performance of the proposed algorithm considering different parameters related to its operation. We also compare it with simple ALOHA and channel-based scheduled allocation and show that the proposed Q-learning based technique has a higher probability of assigning slots compared to these techniques. We then present a block based Q-learning algorithm for the scheduling of MTCDs in clustered M2M communication networks. At first centralized slot assignment is done and an algorithm is proposed for minimizing the inter-cluster interference. Then we propose to use an Q-learning algorithm to assign slots in a distributed manner and comparison is made between the two schemes. Afterwards, we show the effects of distributed slot-assignment with respect to varying signal-to-interference ratio on convergence rate and convergence probability. Cumulative distribution function is used to study the effect of various SIR threshold levels on the convergence probability. With the increase in SIR threshold levels, increase in convergence time and decrease in convergence probability are observed, as less block configuration fulfills the required threshold in the M2M network.

Download Full-text

Distributed Q-Learning Algorithm for Dynamic Resource Allocation With Unknown Objective Functions and Application to Microgrid

IEEE Transactions on Cybernetics ◽

10.1109/tcyb.2021.3082639 ◽

2021 ◽

pp. 1-11

Author(s):

Pengcheng Dai ◽

Wenwu Yu ◽

Duxin Chen

Keyword(s):

Resource Allocation ◽

Learning Algorithm ◽

Dynamic Resource Allocation ◽

Objective Functions ◽

Q Learning ◽

Dynamic Resource

Download Full-text

QoS-based resource allocation for multi-D2D communications in heterogeneous networks

2015 IEEE International Conference on Communication Workshop (ICCW) ◽

10.1109/iccw.2015.7247247 ◽

2015 ◽

Cited By ~ 1

Author(s):

Dong Zheng ◽

Chen He ◽

Lingge Jiang ◽

Jianfeng Ding ◽

Qian Zhang ◽

...

Keyword(s):

Resource Allocation ◽

Heterogeneous Networks ◽

D2d Communications

Download Full-text

Resource allocation in clustered M2M networks: a q-learning approach

10.32920/ryerson.14649174 ◽

2021 ◽

Author(s):

Fatima Hussain

Keyword(s):

Resource Allocation ◽

Learning Algorithm ◽

Random Access ◽

Cumulative Distribution ◽

Convergence Time ◽

Mixed Integer ◽

M2m Communication ◽

Q Learning ◽

Threshold Levels ◽

Slot Assignment

Machine to machine (M2M) communication has received increasing attention in recent years. A M2M network exhibits salient features such as large number of machines/devices, low data rates, delay tolerant/sensitive, small sized packets, energy-constrained and low or no mobility. A large number of M2M terminals may exist in a small area with many trying to simultaneously and randomly access for channel resources - which will result in overload and access problem. This increased signaling overhead and diverse requirements of machine type communication devices (MTCDs) call for the development of flexible and efficient scheduling and random access techniques. In this thesis, we first review and compare various scheduling and random access techniques in LTE-based cellular networks for M2M communication. We also discuss how successful they are to fulfill the unique requirements of M2M communication and networking. Resource management in M2M networks with a large number devices is also reviewed from the access point of view. We propose a multi-objective optimization based solution to the problem of resource allocation in interference-limited M2M communication. We consider MTCDs in a clustered network structure, where they are divided into clusters and the devices belonging to a cluster communicate to cluster head (or controller). We maximize the number of admitted MTCD controllers and throughput with least interference caused to conventional primary users. We formulate the problem as a mixed-integer non-linear problem with multiple objectives and solve it using meshed adaptive direct search (MADS) algorithm. Simulation results show the effects of varying different parameters on cumulative throughput and the number of admitted iii MTCD controllers. We then formulate the slot selection problem in M2M networks with admitted MTCDs as an optimization problem. We present a solution using the Q-learning algorithm to select conflict-free slot assignment in a random access network with MTCD controllers. The performance of the solution is dependent on parameters such as learning rate and reward. We thoroughly analyze the performance of the proposed algorithm considering different parameters related to its operation. We also compare it with simple ALOHA and channel-based scheduled allocation and show that the proposed Q-learning based technique has a higher probability of assigning slots compared to these techniques. We then present a block based Q-learning algorithm for the scheduling of MTCDs in clustered M2M communication networks. At first centralized slot assignment is done and an algorithm is proposed for minimizing the inter-cluster interference. Then we propose to use an Q-learning algorithm to assign slots in a distributed manner and comparison is made between the two schemes. Afterwards, we show the effects of distributed slot-assignment with respect to varying signal-to-interference ratio on convergence rate and convergence probability. Cumulative distribution function is used to study the effect of various SIR threshold levels on the convergence probability. With the increase in SIR threshold levels, increase in convergence time and decrease in convergence probability are observed, as less block configuration fulfills the required threshold in the M2M network.

Download Full-text