Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System

Author(s):  
Yixue Hao ◽  
Min Chen ◽  
Hamid Gharavi ◽  
Yin Zhang ◽  
Kai Hwang
2017 ◽  
Vol 2 (1) ◽  
pp. 44-52
Author(s):  
Kushnir D. ◽  
◽  
Paramud Y.

As a result of the analytical review, it was established that smart sensor units are one of the main components of the cyber–physical system. One of the tasks, which have been entrusted to such units, are targeting and tracking of movable objects. The algorithm of targeting on such objects using observation equipment has been considered. This algorithm is able to continuously monitor observation results, predict the direction with the highest probability of movement and form a set of commands to maximize the approximation of a moving object to the center of an information frame. The algorithm, is based on DDPG reinforcement learning algorithm. The algorithm has been verified on an experimental physical model using a drone. The object recognition module has been developed using YOLOv3 architecture. iOS application has been developed in order to communicate with the drone through WIFI hotspot using UDP commands. Advanced filters have been added to increase the quality of recognition results. The results of experimental research on the mobile platform confirmed the functioning of the targeting algorithm in real–time. Key words: Cyber–physical system, smart sensor unit, reinforcement learning, targeting algorithm, drones.


Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3401
Author(s):  
Ju-Bong Kim ◽  
Ho-Bin Choi ◽  
Gyu-Young Hwang ◽  
Kwihoon Kim ◽  
Yong-Geun Hong ◽  
...  

Intralogistics is a technology that optimizes, integrates, automates, and manages the logistics flow of goods within a logistics transportation and sortation center. As the demand for parcel transportation increases, many sortation systems have been developed. In general, the goal of sortation systems is to route (or sort) parcels correctly and quickly. We design an n-grid sortation system that can be flexibly deployed and used at intralogistics warehouse and develop a collaborative multi-agent reinforcement learning (RL) algorithm to control the behavior of emitters or sorters in the system. We present two types of RL agents, emission agents and routing agents, and they are trained to achieve the given sortation goals together. For the verification of the proposed system and algorithm, we implement them in a full-fledged cyber-physical system simulator and describe the RL agents’ learning performance. From the learning results, we present that the well-trained collaborative RL agents can optimize their performance effectively. In particular, the routing agents finally learn to route the parcels through their optimal paths, while the emission agents finally learn to balance the inflow and outflow of parcels.


Author(s):  
Vo Que Son ◽  
Do Tan A

Sensing, distributed computation and wireless communication are the essential building components of a Cyber-Physical System (CPS). Having many advantages such as mobility, low power, multi-hop routing, low latency, self-administration, utonomous data acquisition, and fault tolerance, Wireless Sensor Networks (WSNs) have gone beyond the scope of monitoring the environment and can be a way to support CPS. This paper presents the design, deployment, and empirical study of an eHealth system, which can remotely monitor vital signs from patients such as body temperature, blood pressure, SPO2, and heart rate. The primary contribution of this paper is the measurements of the proposed eHealth device that assesses the feasibility of WSNs for patient monitoring in hospitals in two aspects of communication and clinical sensing. Moreover, both simulation and experiment are used to investigate the performance of the design in many aspects such as networking reliability, sensing reliability, or end-to-end delay. The results show that the network achieved high reliability - nearly 97% while the sensing reliability of the vital signs can be obtained at approximately 98%. This indicates the feasibility and promise of using WSNs for continuous patient monitoring and clinical worsening detection in general hospital units.


Sign in / Sign up

Export Citation Format

Share Document