Smart Resource Allocation Using Reinforcement Learning in Content-Centric Cyber-Physical Systems

In emerging Cyber-Physical Systems (CPS), the demand for higher communication performance and enhanced wireless connectivity is increasing fast. To address the issue, in our recent work, we proposed a dynamic programming algorithm with polynomial time complexity for effective cross-layer downlink Scheduling and Resource Allocation (SRA) considering the channel and queue state, while supporting fairness. In this paper, we extend the SRA algorithm to consider 5G use-cases, namely enhanced Machine Type Communication (eMTC), Ultra-Reliable Low Latency Communication (URLLC) and enhanced Mobile BroadBand (eMBB). In a simulation study, we evaluate the performance of our SRA algorithm in comparison to an advanced greedy cross-layer algorithm for eMTC, URLLC and LTE (long-term evolution). For eMTC and URLLC, our SRA method outperforms the greedy approach by up to 17.24%, 18.1%, 2.5% and 1.5% in terms of average goodput, correlation impact, goodput fairness and delay fairness, respectively. In the case of LTE, our approach outperforms the greedy method by 60%, 2.6% and 1.6% in terms of goodput, goodput fairness and delay fairness compared with tested baseline.

Download Full-text

Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems

Automatica ◽

10.1016/j.automatica.2019.108759 ◽

2020 ◽

Vol 113 ◽

pp. 108759 ◽

Cited By ~ 9

Author(s):

Alex S. Leong ◽

Arunselvan Ramaswamy ◽

Daniel E. Quevedo ◽

Holger Karl ◽

Ling Shi

Keyword(s):

Reinforcement Learning ◽

Cyber Physical Systems ◽

Wireless Sensor ◽

Sensor Scheduling ◽

Physical Systems

Download Full-text

Adaptive embedded control of cyber-physical systems using reinforcement learning

IET Cyber-Physical Systems Theory & Applications ◽

10.1049/iet-cps.2017.0048 ◽

2017 ◽

Vol 2 (3) ◽

pp. 127-135 ◽

Cited By ~ 1

Author(s):

Hamid Mirzaei Buini ◽

Steffen Peter ◽

Tony Givargis

Keyword(s):

Reinforcement Learning ◽

Cyber Physical Systems ◽

Embedded Control ◽

Physical Systems

Download Full-text

Dynamic Bicycle Dispatching of Dockless Public Bicycle-sharing Systems Using Multi-objective Reinforcement Learning

ACM Transactions on Cyber-Physical Systems ◽

10.1145/3447623 ◽

2021 ◽

Vol 5 (4) ◽

pp. 1-24

Author(s):

Jianguo Chen ◽

Kenli Li ◽

Keqin Li ◽

Philip S. Yu ◽

Zeng Zeng

Keyword(s):

Reinforcement Learning ◽

Dynamic Balance ◽

Supply And Demand ◽

Pareto Frontier ◽

Cyber Physical Systems ◽

Physical Systems ◽

Optimal Dispatch ◽

Multi Objective ◽

Dispatching Algorithm ◽

Multiple Dispatch

As a new generation of Public Bicycle-sharing Systems (PBS), the Dockless PBS (DL-PBS) is an important application of cyber-physical systems and intelligent transportation. How to use artificial intelligence to provide efficient bicycle dispatching solutions based on dynamic bicycle rental demand is an essential issue for DL-PBS. In this article, we propose MORL-BD, a dynamic bicycle dispatching algorithm based on multi-objective reinforcement learning to provide the optimal bicycle dispatching solution for DL-PBS. We model the DL-PBS system from the perspective of cyber-physical systems and use deep learning to predict the layout of bicycle parking spots and the dynamic demand of bicycle dispatching. We define the multi-route bicycle dispatching problem as a multi-objective optimization problem by considering the optimization objectives of dispatching costs, dispatch truck's initial load, workload balance among the trucks, and the dynamic balance of bicycle supply and demand. On this basis, the collaborative multi-route bicycle dispatching problem among multiple dispatch trucks is modeled as a multi-agent and multi-objective reinforcement learning model. All dispatch paths between parking spots are defined as state spaces, and the reciprocal of dispatching costs is defined as a reward. Each dispatch truck is equipped with an agent to learn the optimal dispatch path in the dynamic DL-PBS network. We create an elite list to store the Pareto optimal solutions of bicycle dispatch paths found in each action, and finally get the Pareto frontier. Experimental results on the actual DL-PBS show that compared with existing methods, MORL-BD can find a higher quality Pareto frontier with less execution time.

Download Full-text