THE INTELIGENE ALGORITHM OF CYBER–PHYSICAL SYSTEM TARGETING ON A MOVABLE OBJECT USING THE SMART SENSOR UNIT

2017 ◽  
Vol 2 (1) ◽  
pp. 44-52
Author(s):  
Kushnir D. ◽  
◽  
Paramud Y.
Keyword(s):  
Reinforcement Learning ◽  
Physical System ◽  
Learning Algorithm ◽  
Cyber Physical System ◽  
Smart Sensor ◽  
Sensor Unit ◽  
Movable Object ◽  
Analytical Review ◽  
Main Components

As a result of the analytical review, it was established that smart sensor units are one of the main components of the cyber–physical system. One of the tasks, which have been entrusted to such units, are targeting and tracking of movable objects. The algorithm of targeting on such objects using observation equipment has been considered. This algorithm is able to continuously monitor observation results, predict the direction with the highest probability of movement and form a set of commands to maximize the approximation of a moving object to the center of an information frame. The algorithm, is based on DDPG reinforcement learning algorithm. The algorithm has been verified on an experimental physical model using a drone. The object recognition module has been developed using YOLOv3 architecture. iOS application has been developed in order to communicate with the drone through WIFI hotspot using UDP commands. Advanced filters have been added to increase the quality of recognition results. The results of experimental research on the mobile platform confirmed the functioning of the targeting algorithm in real–time. Key words: Cyber–physical system, smart sensor unit, reinforcement learning, targeting algorithm, drones.

scholarly journals The Algorithm of Cyber-physical System Targeting on a Movable Object Using the Smart Sensor Unit

2017 ◽  
Vol 5 (1) ◽  
pp. 16-22 ◽  
Author(s):  
Dmytro Kushnir ◽  
◽  
Yaroslav Paramud
Keyword(s):  
Object Recognition ◽  
Experimental Research ◽  
Physical System ◽  
Cyber Physical System ◽  
Smart Sensor ◽  
Movable Object ◽  
Main Components ◽  
Information Frame ◽  
Observation Results

It is known that smart sensor units are one of the main components of the cyber-physical system. One of the tasks, which have been entrusted to such units, are targeting and tracking of movable objects. The algorithm of targeting on such objects using observation equipment has been considered. This algorithm is able to continuously monitor observation results, predict the direction with the highest probability of movement and form a set of commands to maximize the approximation of a moving object to the center of an information frame. The algorithm has been verified on an experimental physical model using a drone. The object recognition module has been developed using YOLOv3 architecture. iOS application has been developed in order to communicate with the drone through WIFI hotspot using UDP commands. Advanced filters have been added to increase the quality of recognition results. The results of experimental research on the mobile platform confirmed the functioning of the targeting algorithm in real-time.

scholarly journals Remote health care cyber-physical system: quality of service (QoS) challenges and opportunities

2016 ◽  
Vol 1 (1) ◽  
pp. 40-48 ◽  
Author(s):  
Tejal Shah ◽  
Ali Yavari ◽  
Karan Mitra ◽  
Saguna Saguna ◽  
Prem Prakash Jayaraman ◽  
...  
Keyword(s):  
Health Care ◽  
Quality Of Service ◽  
Physical System ◽  
Cyber Physical System ◽  
System Quality ◽  
Challenges And Opportunities ◽  
Remote Health

scholarly journals Categorization of material quality using a model-free reinforcement learning algorithm

2021 ◽  
Vol 2107 (1) ◽  
pp. 012027
Author(s):  
Annapoorni Mani ◽  
Shahriman Abu Bakar ◽  
Pranesh Krishnan ◽  
Sazali Yaacob
Keyword(s):  
Reinforcement Learning ◽  
Optimization Problems ◽  
Learning Algorithm ◽  
Raw Materials ◽  
Model Free ◽  
Ideal Approach ◽  
Inspection Process ◽  
The Ideal ◽  
Incoming Inspection

Abstract Reinforcement learning is the most preferred algorithms for optimization problems in industrial automation. Model-free reinforcement learning algorithms optimize for rewards without the knowledge of the environmental dynamics and require less computation. Regulating the quality of the raw materials in the inbound inventory can improve the manufacturing process. In this paper, the raw materials arriving at the incoming inspection process are categorized and labeled based on their quality through the path traveled. A model-free temporal difference learning approach is used to predict the acceptance and rejection path of raw materials in the incoming inspection process. The algorithm presented eight routes paths that the raw materials could travel. Four pathways correspond to material acceptance, while the rest lead to material refusal. The materials are annotated using the total scores acquired in the incoming inspection process. The materials traveling on the ideal path (path A) get the highest total score. The rest of the accepted materials in the acceptance path have a 7.37% lower score in path B, whereas path C and path D get 37.28% and 42.44% lower than the ideal approach.

scholarly journals Integrating Physics and Data Driven Cyber-Physical System for Condition Monitoring of Critical Transmission Components in Smart Production Line

2021 ◽  
Vol 11 (19) ◽  
pp. 8967
Author(s):  
Lin Song ◽  
Liping Wang ◽  
Jun Wu ◽  
Jianhong Liang ◽  
Zhigui Liu
Keyword(s):  
Signal Processing ◽  
Deep Learning ◽  
Condition Monitoring ◽  
Physical System ◽  
Production Line ◽  
Learning Algorithm ◽  
Data Driven ◽  
Equation Modeling ◽  
Cyber Physical System ◽  
Deep Learning Algorithm

In response to the lack of a unified cyber–physical system framework, which combined the Internet of Things, industrial big data, and deep learning algorithms for the condition monitoring of critical transmission components in a smart production line. In this study, based on the conceptualization of the layers, a novel five-layer cyber–physical systems framework for smart production lines is proposed. This architecture integrates physics and is data-driven. The smart connection layer collects and transmits data, the physical equation modeling layer converts low-value raw data into high-value feature information via signal processing, the machine learning modeling layer realizes condition prediction through a deep learning algorithm, and scientific decision-making and predictive maintenance are completed through a cognition layer and a configuration layer. Case studies on three critical transmission components—spindles, bearings, and gears—are carried out to validate the effectiveness of the proposed framework and hybrid model for condition monitoring. The prediction results of the three datasets show that the system is successful in distinguishing condition, while the short time Fourier transform signal processing and deep residual network deep learning algorithm is superior to that of other models. The proposed framework and approach are scalable and generalizable and lay the foundation for the extension of the model.

scholarly journals Relative control of an underactuated spacecraft using reinforcement learning

2020 ◽  
Vol 2020 (4) ◽  
pp. 43-54
Author(s):  
S.V. Khoroshylov ◽  
◽  
M.O. Redka ◽  
Keyword(s):  
Neural Network ◽  
Control System ◽  
Reinforcement Learning ◽  
Control Theory ◽  
Learning Algorithm ◽  
Control Algorithms ◽  
Iteration Algorithm ◽  
Quality Of Control ◽  
Control Actions

The aim of the article is to approximate optimal relative control of an underactuated spacecraft using reinforcement learning and to study the influence of various factors on the quality of such a solution. In the course of this study, methods of theoretical mechanics, control theory, stability theory, machine learning, and computer modeling were used. The problem of in-plane spacecraft relative control using only control actions applied tangentially to the orbit is considered. This approach makes it possible to reduce the propellant consumption of reactive actuators and to simplify the architecture of the control system. However, in some cases, methods of the classical control theory do not allow one to obtain acceptable results. In this regard, the possibility of solving this problem by reinforcement learning methods has been investigated, which allows designers to find control algorithms close to optimal ones as a result of interactions of the control system with the plant using a reinforcement signal characterizing the quality of control actions. The well-known quadratic criterion is used as a reinforcement signal, which makes it possible to take into account both the accuracy requirements and the control costs. A search for control actions based on reinforcement learning is made using the policy iteration algorithm. This algorithm is implemented using the actor–critic architecture. Various representations of the actor for control law implementation and the critic for obtaining value function estimates using neural network approximators are considered. It is shown that the optimal control approximation accuracy depends on a number of features, namely, an appropriate structure of the approximators, the neural network parameter updating method, and the learning algorithm parameters. The investigated approach makes it possible to solve the considered class of control problems for controllers of different structures. Moreover, the approach allows the control system to refine its control algorithms during the spacecraft operation.

scholarly journals Reliability-Aware: Task Scheduling in Cloud Computing Using Multi-Agent Reinforcement Learning Algorithm and Neural Fitted Q

2020 ◽  
Vol 18 (1) ◽  
Keyword(s):  
Cloud Computing ◽  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Buffer Size ◽  
Cloud Environment ◽  
Numerical Result ◽  
Learning Agent ◽  
Multi Agent ◽  
Reinforcement Learning Algorithm

Cloud computing becomes the basic alternative platform for the most users application in the recent years. The complexity increasing in cloud environment due to the continuous development of resources and applications needs a concentrated integrated fault tolerance approach to provide the quality of service. Focusing on reliability enhancement in an environment with dynamic changes such as cloud environment, we developed a multi-agent scheduler using Reinforcement Learning (RL) algorithm and Neural Fitted Q (NFQ) to effectively schedule the user requests. Our approach considers the queue buffer size for each resource by implementing the queue theory to design a queue model in a way that each scheduler agent has its own queue which receives the user requests from the global queue. A central learning agent responsible of learning the output of the scheduler agents and direct those scheduler agents through the feedback claimed from the previous step. The dynamicity problem in cloud environment is managed in our system by employing neural network which supports the reinforcement learning algorithm through a specified function. The numerical result demonstrated an efficiency of our proposed approach and enhanced the reliability

Sign in / Sign up

Export Citation Format

Share Document