Deep Reinforcement Learning for Mobile Video Offloading in Heterogeneous Cellular Networks

2018 ◽  
Vol 9 (4) ◽  
pp. 34-57 ◽  
Author(s):  
Nan Zhao ◽  
Chao Tian ◽  
Menglin Fan ◽  
Minghu Wu ◽  
Xiao He ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Cellular Networks ◽  
Communication Networks ◽  
Optimal Solution ◽  
Mobile Video ◽  
Heterogeneous Cellular Networks ◽  
Optimal Method ◽  
Q Learning ◽  
Mobile Offloading

Heterogeneous cellular networks can balance mobile video loads and reduce cell arrangement costs, which is an important technology of future mobile video communication networks. Because of the characteristics of non-convexity of the mobile offloading problem, the design of the optimal strategy is an essential issue. For the sake of ensuring users' quality of service and the long-term overall network utility, this article proposes the distributive optimal method by means of multiple agent reinforcement learning in the downlink heterogeneous cellular networks. In addition, to solve the computational load issue generated by the large action space, deep reinforcement learning is introduced to gain the optimal policy. The learning policy can provide a near-optimal solution efficiently with a fast convergence speed. Simulation results show that the proposed approach is more efficient at improving the performance than the Q-learning method.

scholarly journals Aircraft Maintenance Check Scheduling Using Reinforcement Learning

Aerospace ◽  
2021 ◽  
Vol 8 (4) ◽  
pp. 113
Author(s):  
Pedro Andrade ◽  
Catarina Silva ◽  
Bernardete Ribeiro ◽  
Bruno F. Santos
Keyword(s):  
Reinforcement Learning ◽  
Time Horizon ◽  
Learning Algorithm ◽  
Initial Conditions ◽  
Q Learning ◽  
Scheduling Policy ◽  
Real Scenario ◽  
Maintenance Plan ◽  
Small Disturbances

This paper presents a Reinforcement Learning (RL) approach to optimize the long-term scheduling of maintenance for an aircraft fleet. The problem considers fleet status, maintenance capacity, and other maintenance constraints to schedule hangar checks for a specified time horizon. The checks are scheduled within an interval, and the goal is to, schedule them as close as possible to their due date. In doing so, the number of checks is reduced, and the fleet availability increases. A Deep Q-learning algorithm is used to optimize the scheduling policy. The model is validated in a real scenario using maintenance data from 45 aircraft. The maintenance plan that is generated with our approach is compared with a previous study, which presented a Dynamic Programming (DP) based approach and airline estimations for the same period. The results show a reduction in the number of checks scheduled, which indicates the potential of RL in solving this problem. The adaptability of RL is also tested by introducing small disturbances in the initial conditions. After training the model with these simulated scenarios, the results show the robustness of the RL approach and its ability to generate efficient maintenance plans in only a few seconds.

Stackelberg Game Theoretic Framework in Cognitive Green Heterogeneous Networks

2017 ◽  
pp. 269-286
Author(s):  
Chungang Yang ◽  
Pengyu Huang ◽  
Jia Xiao ◽  
Lingxia Wang ◽  
Jiandong Li
Keyword(s):  
Decision Making ◽  
Energy Consumption ◽  
Cognitive Radio ◽  
Cellular Networks ◽  
Communication Networks ◽  
Cognitive Radio Networks ◽  
Stackelberg Game ◽  
Radio Networks ◽  
Heterogeneous Cellular Networks ◽  
Game Theoretic

Game theory has found an extensive application in wireless communication networks including cognitive radio networks, heterogeneous cellular networks, cooperative relay networks. Also, cognitive radio networks, green communications and heterogeneous cellular networks have attracted a wide attention on improve the spectrum efficiency and energy efficiency; therefore, the capacity, the coverage and the energy consumption. However, interference problem and energy consumption are critical for these networks. Introducing hierarchy among different decision-making players in cognitive, heterogeneous, green, cooperative cellular networks can both save energy and mitigate interference, thus enhance throughput. Stackelberg game suits to model, analyze and design the distributed algorithms in these hierarchical decision-making networking scenarios. In this chapter, we introduce basics of Stackelberg game and survey the extensive applications of Stackelberg game in cognitive, heterogeneous, cooperative cellular networks with the emphasis on resource management, green commutations design and interference management. This chapter highlights the potentials and applications with the promising vision of Stackelberg game theoretic framework for future cognitive green heterogeneous cellular networks.

scholarly journals Joint User Association and Energy Offloading in Downlink Heterogeneous Cellular Networks

2018 ◽  
Vol 2018 ◽  
pp. 1-9 ◽  
Author(s):  
Rui Li ◽  
Ning Cao ◽  
Minghe Mao ◽  
Yunfei Chen ◽  
Yifan Hu
Keyword(s):  
Power Allocation ◽  
Cellular Networks ◽  
Communication Systems ◽  
System Throughput ◽  
User Association ◽  
Heterogeneous Cellular Networks ◽  
Macro Cell ◽  
System Resource ◽  
Resource Allocation Scheme

As a key technology in Long-Term Evolution-Advanced (LTE-A) mobile communication systems, heterogeneous cellular networks (HCNs) add low-power nodes to offload the traffic from macro cell and therefore improve system throughput performance. In this paper, we investigate a joint user association and resource allocation scheme for orthogonal frequency division multiple access- (OFDMA-) based downlink HCNs for maximizing the energy efficiency and optimizing the system resource. The algorithm is formulated as a nonconvex optimization, with dynamic circuit consumption, limited transmit power, and quality-of-service (QoS) constraints. As a nonlinear fractional problem, an iteration-based algorithm is proposed to decompose the problem into two subproblems, that is, user association and power allocation. For each iteration, we alternatively solve the two subproblems and obtain the optimal user association and power allocation strategies. Numerical results illustrate that the proposed iteration-based algorithm outperforms existing algorithms.

scholarly journals A Q-Learning-Based Approach for Deploying Dynamic Service Function Chains

Symmetry ◽  
2018 ◽  
Vol 10 (11) ◽  
pp. 646 ◽  
Author(s):  
Jian Sun ◽  
Guanhua Huang ◽  
Gang Sun ◽  
Hongfang Yu ◽  
Arun Kumar Sangaiah ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Service Provider ◽  
Heuristic Algorithms ◽  
Dynamic Networks ◽  
Dynamic Network ◽  
Optimal Solution ◽  
Network Function Virtualization ◽  
Time Efficiency ◽  
Q Learning ◽  
Service Function

As the size and service requirements of today’s networks gradually increase, large numbers of proprietary devices are deployed, which leads to network complexity, information security crises and makes network service and service provider management increasingly difficult. Network function virtualization (NFV) technology is one solution to this problem. NFV separates network functions from hardware and deploys them as software on a common server. NFV can be used to improve service flexibility and isolate the services provided for each user, thus guaranteeing the security of user data. Therefore, the use of NFV technology includes many problems worth studying. For example, when there is a free choice of network path, one problem is how to choose a service function chain (SFC) that both meets the requirements and offers the service provider maximum profit. Most existing solutions are heuristic algorithms with high time efficiency, or integer linear programming (ILP) algorithms with high accuracy. It’s necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. In this paper, we propose the Q-learning Framework Hybrid Module algorithm (QLFHM), which includes reinforcement learning to solve this SFC deployment problem in dynamic networks. The reinforcement learning module in QLFHM is responsible for the output of alternative paths, while the load balancing module in QLFHM is responsible for picking the optimal solution from them. The results of a comparison simulation experiment on a dynamic network topology show that the proposed algorithm can output the approximate optimal solution in a relatively short time while also considering the network load balance. Thus, it achieves the goal of maximizing the benefit to the service provider.

Multi-Agent Actor Critic for Channel Allocation in Heterogeneous Networks

2020 ◽  
Vol 11 (1) ◽  
pp. 23-41
Author(s):  
Nan Zhao ◽  
Zehua Liu ◽  
Yiqiang Cheng ◽  
Chao Tian
Keyword(s):  
Reinforcement Learning ◽  
Communication Networks ◽  
Heterogeneous Networks ◽  
High Efficiency ◽  
Channel Allocation ◽  
Optimal Solution ◽  
New Method ◽  
Multi Agent ◽  
Optimal Approach ◽  
The Cost

Heterogeneous networks (HetNets) can equalize traffic loads and cut down the cost of deploying cells. Thus, it is regarded to be the significant technique of the next-generation communication networks. Due to the non-convexity nature of the channel allocation problem in HetNets, it is difficult to design an optimal approach for allocating channels. To ensure the user quality of service as well as the long-term total network utility, this article proposes a new method through utilizing multi-agent reinforcement learning. Moreover, for the purpose of solving computational complexity problem caused by the large action space, deep reinforcement learning is put forward to learn optimal policy. A nearly-optimal solution with high efficiency and rapid convergence speed could be obtained by this learning method. Simulation results reveal that this new method has the best performance than other methods.

Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks

2019 ◽  
Vol 18 (11) ◽  
pp. 5141-5152 ◽  
Author(s):  
Nan Zhao ◽  
Ying-Chang Liang ◽  
Dusit Niyato ◽  
Yiyang Pei ◽  
Minghu Wu ◽  
...  
Keyword(s):  
Resource Allocation ◽  
Reinforcement Learning ◽  
Cellular Networks ◽  
User Association ◽  
Heterogeneous Cellular Networks

scholarly journals A Practical Tessellation-Based Approach for Optimizing Cell-Specific Bias Values in LTE-A Heterogeneous Cellular Networks

2017 ◽  
Vol 2017 ◽  
pp. 1-11
Author(s):  
Jin-Bae Park ◽  
Kwang Soon Kim
Keyword(s):  
Load Balancing ◽  
Cellular Networks ◽  
User Association ◽  
Heterogeneous Cellular Networks ◽  
Cell Range Expansion ◽  
Intercell Interference ◽  
Core Networks ◽  
Almost Blank Subframe ◽  
Better Than

In order to implement an optimized solution for cell range expansion (CRE) and enhanced intercell interference coordination (eICIC) schemes in long-term evolution-advanced (LTE-A) heterogeneous cellular networks (HCNs) and to realize good load-balancing performance in existing LTE-A systems, a practical tessellation-based algorithm is proposed. In this algorithm, a globalized cell-specific bias optimization and a localized almost blank subframe (ABS) ratio update are proposed. The proposed scheme does not require major changes to existing protocols. Thus, it can be implemented in existing LTE-A systems with any legacy user equipment (UE) with only a partial update to the BSs and core networks. From simulation results, it is shown that the tessellation formed by the proposed approach is quite consistent with the optimal one for various realistic scenarios. Thus, the proposed scheme can provide a much better load-balancing capability compared with the conventional common bias scheme. Owing to the improved load-balancing capability, the user rate distribution of the proposed scheme is much better than that obtained from the conventional scheme and is even indistinguishable from that of the ideal joint user association scheme.

Sign in / Sign up

Export Citation Format

Share Document