scholarly journals Multi-agent deep reinforcement learning concept for mobile cyber-physical systems control

2021 ◽  
Vol 270 ◽  
pp. 01036
Author(s):  
Vyacheslav Petrenko ◽  
Mikhail Gurchinskiy

High complexity of mobile cyber physical systems (MCPS) dynamics makes it difficult to apply classical methods to optimize the MCPS agent management policy. In this regard, the use of intelligent control methods, in particular, with the help of artificial neural networks (ANN) and multi-agent deep reinforcement learning (MDRL), is gaining relevance. In practice, the application of MDRL in MCPS faces the following problems: 1) existing MDRL methods have low scalability; 2) the inference of the used ANNs has high computational complexity; 3) MCPS trained using existing methods have low functional safety. To solve these problems, we propose the concept of a new MDRL method based on the existing MADDPG method. Within the framework of the concept, it is proposed: 1) to increase the scalability of MDRL by using information not about all other MCPS agents, but only about n nearest neighbors; 2) reduce the computational complexity of ANN inference by using a sparse ANN structure; 3) to increase the functional safety of trained MCPS by using a training set with uneven distribution of states. The proposed concept is expected to help address the challenges of applying MDRL to MCPS. To confirm this, it is planned to conduct experimental studies.

2020 ◽  
Vol 10 (9) ◽  
pp. 3125
Author(s):  
Saad Mubeen ◽  
Elena Lisova ◽  
Aneta Vulgarakis Feljan

Cyber Physical Systems (CPSs) are systems that are developed by seamlessly integrating computational algorithms and physical components, and they are a result of the technological advancement in the embedded systems and distributed systems domains, as well as the availability of sophisticated networking technology. Many industrial CPSs are subject to timing predictability, security and functional safety requirements, due to which the developers of these systems are required to verify these requirements during the their development. This position paper starts by exploring the state of the art with respect to developing timing predictable and secure embedded systems. Thereafter, the paper extends the discussion to time-critical and secure CPSs and highlights the key issues that are faced when verifying the timing predictability requirements during the development of these systems. In this context, the paper takes the position to advocate paramount importance of security as a prerequisite for timing predictability, as well as both security and timing predictability as prerequisites for functional safety. Moreover, the paper identifies the gaps in the existing frameworks and techniques for the development of time- and safety-critical CPSs and describes our viewpoint on ensuring timing predictability and security in these systems. Finally, the paper emphasises the opportunities that artificial intelligence can provide in the development of these systems.


Automatica ◽  
2020 ◽  
Vol 113 ◽  
pp. 108759 ◽  
Author(s):  
Alex S. Leong ◽  
Arunselvan Ramaswamy ◽  
Daniel E. Quevedo ◽  
Holger Karl ◽  
Ling Shi

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 58810-58821 ◽  
Author(s):  
Youwei Yuan ◽  
Jintao Zhang ◽  
Guangjie Han ◽  
Gangyong Jia ◽  
Lamei Yan ◽  
...  

2020 ◽  
Vol 51 ◽  
pp. 1200-1206
Author(s):  
Guolin Lyu ◽  
Alireza Fazlirad ◽  
Robert W. Brennan

2015 ◽  
Vol 2015 ◽  
pp. 1-13
Author(s):  
Yiqi Zhang ◽  
Changxu Wu ◽  
Chunming Qiao ◽  
Adel Sadek ◽  
Kevin F. Hulme

As an important application of Cyber-Physical Systems (CPS), advances in intelligent transportation systems (ITS) improve driving safety by informing drivers of hazards with warnings in advance. The evaluation of the warning effectiveness is an important issue in facilitating communication of ITS. The goal of the present study was to develop a scale to evaluate the warning utility, namely, the effectiveness of a warning in preventing accidents in general. A driving simulator study was conducted to validate the Verbal Warning Utility Scale (VWUS) in a simulated driving environment. The reliability analysis indicated a good split-half reliability for the VWUS with a Spearman-Brown Coefficient of 0.873. The predictive validity of VWUS in measuring the effectiveness of the verbal warnings was verified by the significant prediction of safety benefits indicated by variables, including reduced kinetic energy and collision rate. Compared to conducting experimental studies, this scale provides a simpler way to evaluate overall utility of verbal warnings in communicating associated hazards in intelligent transportation systems. This scale can be further applied to improve the design of warnings of ITS in order to improve transportation safety. The applications of the scale in nonverbal warning situations and limitations of the current scale are also discussed.


Sign in / Sign up

Export Citation Format

Share Document