Controlling a cargo ship without human experience using deep Q-network

2020 ◽  
Vol 39 (5) ◽  
pp. 7363-7379
Author(s):  
Chen Chen ◽  
Feng Ma ◽  
Jialun Liu ◽  
Rudy R. Negenborn ◽  
Yuanchang Liu ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Decision Making ◽  
Potential Field ◽  
Driving Force ◽  
Human Experience ◽  
Artificial Potential Field ◽  
Value Functions ◽  
Simulation Experiments ◽  
Promising Solution ◽  
Better Than

Human experience is regarded as an indispensable part of artificial intelligence in the process of controlling or decision making for autonomous cargo ships. In this paper, a novel Deep Q-Network-based (DQN) approach is proposed, which performs satisfactorily in controlling a cargo ship automatically without any human experience. At the very beginning, we use the model of KRISO Very Large Crude Carrier (KVLCC2) to describe a cargo ship. To manipulate this ship has to conquer great inertia and relatively insufficient driving force. Subsequently, customary waterways, regulations, conventions are described with Artificial Potential Field and value-functions in DQN. Based on this, the artificial intelligence of planning and controlling a cargo ship can be obtained by undertaking sufficient training, which can control the ship directly, while avoiding collisions, keeping its position in the middle of the route as much as possible. In simulation experiments, it is demonstrated that such an approach performs better than manual works and other traditional methods in most conditions, which makes the proposed method a promising solution in improving the autonomy level of cargo ships.

Adaptation and Customization in Virtual Rehabilitation

2016 ◽  
pp. 141-163 ◽  
Author(s):  
Felipe Orihuela-Espina ◽  
L. Enrique Sucar
Keyword(s):  
Artificial Intelligence ◽  
Decision Making ◽  
Driving Force ◽  
Therapy Outcome ◽  
Motor Training ◽  
Virtual Rehabilitation ◽  
Potential Benefits ◽  
The Many ◽  
System Configurations ◽  
Privacy Issues

Background. Adaptation and customization are two related but distinct concepts that are central to virtual rehabilitation if this motor therapy modality is to succeed in alleviating the demand for expert supervision. These two elements of the therapy are required to exploit the flexibility of virtual environments to enhance motor training and boost therapy outcome. Aim. The chapter provides a non-systematic overview of the state of the art regarding the evolving manipulation of virtual rehabilitation environments to optimize therapy outcome manifested through customization and adaptation mechanisms. Methods. Both concepts will be defined, aspects guiding their implementation reviewed, and available literature suggesting different solutions discussed. We present “Gesture Therapy”, a platform realizing our contributions to the field and we present results of the adaptation techniques integrated into it. Less explored additional dimensions such as liability and privacy issues affecting their implementation are briefly discussed. Results. Solutions to implement decision-making on how to manipulate the environment are varied. They range from predefined system configurations to sophisticated artificial intelligence (AI) models. Challenge maintenance and feedback personalization is the most common driving force for their incorporation to virtual rehabilitation platforms. Conclusions. Customization and adaptation are the main mechanisms responsible for the full exploitation of the potential of virtual rehabilitation environments, and the potential benefits are worth pursuing. Despite encouraging evidence of the many solutions proposed thus far in literature, none has yet proven to substantially alter the therapy outcome. In consequence, research is still on going to equip virtual rehabilitation solutions with efficacious tailoring elements.

The Simulation of the UAV Collision Avoidance Based on the Artificial Potential Field Method

2012 ◽  
Vol 591-593 ◽  
pp. 1400-1404 ◽  
Author(s):  
Jian Ying Liu ◽  
Zi Qi Guo ◽  
Shi Yue Liu
Keyword(s):  
Potential Field ◽  
Field Method ◽  
Artificial Potential Field ◽  
Lyapunov Theorem ◽  
Balance Point ◽  
Dynamic Path Planning ◽  
Simulation Results ◽  
Dynamic Path ◽  
Artificial Potential Field Method ◽  
Better Than

To deal with dynamic path planning of unmanned aerial vehicles(uav) in the complicated 3-D environment, a new method that combines the Lyapunov theorem with the artificial potential is proposed. The mission region is described as the artificial potential field. In this paper, it proves that the balance point is a saddle point, only when uav reaches the target, the balance point is stable, the rest of the balance point are divergent, so uav can escape the minimum point as soon as possible. The simulation results show that this proposed method can effectively make uav avoid collision, and escape well the local minimum value point. The optimization results are better than the simplex artificial potential field, and have better optimization precision and tracing speed.

Adaptation and Customization in Virtual Rehabilitation

2018 ◽  
pp. 826-849
Author(s):  
Felipe Orihuela-Espina ◽  
L. Enrique Sucar
Keyword(s):  
Artificial Intelligence ◽  
Decision Making ◽  
Driving Force ◽  
Therapy Outcome ◽  
Motor Training ◽  
Virtual Rehabilitation ◽  
Potential Benefits ◽  
The Many ◽  
System Configurations ◽  
Privacy Issues

Background. Adaptation and customization are two related but distinct concepts that are central to virtual rehabilitation if this motor therapy modality is to succeed in alleviating the demand for expert supervision. These two elements of the therapy are required to exploit the flexibility of virtual environments to enhance motor training and boost therapy outcome. Aim. The chapter provides a non-systematic overview of the state of the art regarding the evolving manipulation of virtual rehabilitation environments to optimize therapy outcome manifested through customization and adaptation mechanisms. Methods. Both concepts will be defined, aspects guiding their implementation reviewed, and available literature suggesting different solutions discussed. We present “Gesture Therapy”, a platform realizing our contributions to the field and we present results of the adaptation techniques integrated into it. Less explored additional dimensions such as liability and privacy issues affecting their implementation are briefly discussed. Results. Solutions to implement decision-making on how to manipulate the environment are varied. They range from predefined system configurations to sophisticated artificial intelligence (AI) models. Challenge maintenance and feedback personalization is the most common driving force for their incorporation to virtual rehabilitation platforms. Conclusions. Customization and adaptation are the main mechanisms responsible for the full exploitation of the potential of virtual rehabilitation environments, and the potential benefits are worth pursuing. Despite encouraging evidence of the many solutions proposed thus far in literature, none has yet proven to substantially alter the therapy outcome. In consequence, research is still on going to equip virtual rehabilitation solutions with efficacious tailoring elements.

scholarly journals An obstacles avoidance method for serial manipulator based on reinforcement learning and Artificial Potential Field

2021 ◽  
Author(s):  
Haoxuan Li ◽  
Daoxiong Gong ◽  
Jianjun Yu
Keyword(s):  
Reinforcement Learning ◽  
Potential Field ◽  
Avoidance Performance ◽  
Field Method ◽  
Artificial Potential Field ◽  
Simulation Experiments ◽  
Reward Function ◽  
Physical Experiments ◽  
Different Types ◽  
Artificial Potential Field Method

AbstractThe obstacles avoidance of manipulator is a hot issue in the field of robot control. Artificial Potential Field Method (APFM) is a widely used obstacles avoidance path planning method, which has prominent advantages. However, APFM also has some shortcomings, which include the inefficiency of avoiding obstacles close to target or dynamic obstacles. In view of the shortcomings of APFM, Reinforcement Learning (RL) only needs an automatic learning model to continuously improve itself in the specified environment, which makes it capable of optimizing APFM theoretically. In this paper, we introduce an approach hybridizing RL and APFM to solve those problems. We define the concepts of Distance reinforcement factors (DRF) and Force reinforcement factors (FRF) to make RL and APFM integrated more effectively. We disassemble the reward function of RL into two parts through DRF and FRF, and make them activate in different situations to optimize APFM. Our method can obtain better obstacles avoidance performance through finding the optimal strategy by RL, and the effectiveness of the proposed algorithm is verified by multiple sets of simulation experiments, comparative experiments and physical experiments in different types of obstacles. Our approach is superior to traditional APFM and the other improved APFM method in avoiding collisions and approaching obstacles avoidance. At the same time, physical experiments verify the practicality of the proposed algorithm.

Beérkezett könyvek

2008 ◽  
Vol 63 (3) ◽  
pp. 607-608
Author(s):  
Csaba Pléh
Keyword(s):  
Decision Making ◽  
Human Mind ◽  
Frankfurt Am Main ◽  
Conscious Decision ◽  
Better Than

ErősFerenc, LénárdKataés BókayAntal(szerk.) Typus Budapestiensis. Tanulmányok a pszichoanalízis budapesti iskolájának történetéről éshatásáról. Thalassa, Budapest, 2008, 447 oldalHargittaiIstván: Doktor DNS. Őszinte beszélgetések James D. Watsonnal. Vince Kiadó, Budapest, 2008, 223 oldalKutrovátzGábor,LángBenedekésZemplénGábor: A tudomány határa. Typotex,Budapest, 2008, 376 oldalEngerl, C. andSinger, W. (eds) Better than conscious? Decision making, the human mind, and implications for institutions . MIT Press, Cambridge, 2008, xiv + 449 oldalKondor, Zsuzsanna: Embedded thinking. Multimedia and the new rationality. Peter Lang, Frankfurt am Main, 2008, xi + 169 oldalSíklakiIstván(szerk.): Szóbeli befolyásolás. I–II. Typotex, Budapest,_n

Sign in / Sign up

Export Citation Format

Share Document