scholarly journals Application of Deep Reinforcement Learning to Predict Shaft Deformation Considering Hull Deformation of Medium-Sized Oil/Chemical Tanker

2021 ◽  
Vol 9 (7) ◽  
pp. 767
Author(s):  
Shin-Pyo Choi ◽  
Jae-Ung Lee ◽  
Jun-Bum Park
Keyword(s):  
Reinforcement Learning ◽  
Reaction Force ◽  
Measurement Data ◽  
System Stability ◽  
Main Bearing ◽  
Shaft System ◽  
Novel Approach ◽  
Design Changes ◽  
Prediction Techniques ◽  
Shaft Deformation

The enlargement of ships has increased the relative hull deformation owing to draft changes. Moreover, design changes such as an increased propeller diameter and pitch changes have occurred to compensate for the reduction in the engine revolution and consequent ship speed. In terms of propulsion shaft alignment, as the load of the stern tube support bearing increases, an uneven load distribution occurs between the shaft support bearings, leading to stern accidents. To prevent such accidents and to ensure shaft system stability, a shaft system design technique is required in which the shaft deformation resulting from the hull deformation is considered. Based on the measurement data of a medium-sized oil/chemical tanker, this study presents a novel approach to predicting the shaft deformation following stern hull deformation through inverse analysis using deep reinforcement learning, as opposed to traditional prediction techniques. The main bearing reaction force, which was difficult to reflect in previous studies, was predicted with high accuracy by comparing it with the measured value, and reasonable shaft deformation could be derived according to the hull deformation. The deep reinforcement learning technique in this study is expected to be expandable for predicting the dynamic behavior of the shaft of an operating vessel.

Teaching AI Agents Ethical Values Using Reinforcement Learning and Policy Orchestration

2019 ◽  
Author(s):  
Ritesh Noothigattu ◽  
Djallel Bouneffouf ◽  
Nicholas Mattei ◽  
Rachita Chandra ◽  
Piyush Madan ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Ethical Values ◽  
Large Role ◽  
Learning To Learn ◽  
Inverse Reinforcement Learning ◽  
Time Step ◽  
Novel Approach

Autonomous cyber-physical agents play an increasingly large role in our lives. To ensure that they behave in ways aligned with the values of society, we must develop techniques that allow these agents to not only maximize their reward in an environment, but also to learn and follow the implicit constraints of society. We detail a novel approach that uses inverse reinforcement learning to learn a set of unspecified constraints from demonstrations and reinforcement learning to learn to maximize environmental rewards. A contextual bandit-based orchestrator then picks between the two policies: constraint-based and environment reward-based. The contextual bandit orchestrator allows the agent to mix policies in novel ways, taking the best actions from either a reward-maximizing or constrained policy. In addition, the orchestrator is transparent on which policy is being employed at each time step. We test our algorithms using Pac-Man and show that the agent is able to learn to act optimally, act within the demonstrated constraints, and mix these two functions in complex ways.

scholarly journals A Novel Approach to Feedback Control with Deep Reinforcement Learning

2018 ◽  
Vol 51 (18) ◽  
pp. 31-36 ◽  
Author(s):  
Yuan Wang ◽  
Kirubakaran Velswamy ◽  
Biao Huang
Keyword(s):  
Reinforcement Learning ◽  
Feedback Control ◽  
Novel Approach

scholarly journals Crowd Evacuation Guidance Based on Combined Action Reinforcement Learning

Algorithms ◽  
2021 ◽  
Vol 14 (1) ◽  
pp. 26
Author(s):  
Yiran Xue ◽  
Rui Wu ◽  
Jiafeng Liu ◽  
Xianglong Tang
Keyword(s):  
Reinforcement Learning ◽  
Guidance System ◽  
Force Model ◽  
Interactive Simulation ◽  
Social Force ◽  
Novel Approach ◽  
Learning Agent ◽  
Network Output ◽  
Combined Action ◽  
Crowd Evacuation

Existing crowd evacuation guidance systems require the manual design of models and input parameters, incurring a significant workload and a potential for errors. This paper proposed an end-to-end intelligent evacuation guidance method based on deep reinforcement learning, and designed an interactive simulation environment based on the social force model. The agent could automatically learn a scene model and path planning strategy with only scene images as input, and directly output dynamic signage information. Aiming to solve the “dimension disaster” phenomenon of the deep Q network (DQN) algorithm in crowd evacuation, this paper proposed a combined action-space DQN (CA-DQN) algorithm that grouped Q network output layer nodes according to action dimensions, which significantly reduced the network complexity and improved system practicality in complex scenes. In this paper, the evacuation guidance system is defined as a reinforcement learning agent and implemented by the CA-DQN method, which provides a novel approach for the evacuation guidance problem. The experiments demonstrate that the proposed method is superior to the static guidance method, and on par with the manually designed model method.

scholarly journals Deep Reinforcement Learning for End-to-End Local Motion Planning of Autonomous Aerial Robots in Unknown Outdoor Environments: Real-Time Flight Experiments

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2534
Author(s):  
Oualid Doukhi ◽  
Deok-Jin Lee
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Autonomous Navigation ◽  
Control Technique ◽  
Local Motion ◽  
Aerial Robots ◽  
Novel Approach ◽  
Outdoor Environments ◽  
Aerial Vehicle ◽  
High Level

Autonomous navigation and collision avoidance missions represent a significant challenge for robotics systems as they generally operate in dynamic environments that require a high level of autonomy and flexible decision-making capabilities. This challenge becomes more applicable in micro aerial vehicles (MAVs) due to their limited size and computational power. This paper presents a novel approach for enabling a micro aerial vehicle system equipped with a laser range finder to autonomously navigate among obstacles and achieve a user-specified goal location in a GPS-denied environment, without the need for mapping or path planning. The proposed system uses an actor–critic-based reinforcement learning technique to train the aerial robot in a Gazebo simulator to perform a point-goal navigation task by directly mapping the noisy MAV’s state and laser scan measurements to continuous motion control. The obtained policy can perform collision-free flight in the real world while being trained entirely on a 3D simulator. Intensive simulations and real-time experiments were conducted and compared with a nonlinear model predictive control technique to show the generalization capabilities to new unseen environments, and robustness against localization noise. The obtained results demonstrate our system’s effectiveness in flying safely and reaching the desired points by planning smooth forward linear velocity and heading rates.

PEBB System Stability Margin Monitoring

2002 ◽  
Vol 8 (2) ◽  
pp. 261-276 ◽  
Author(s):  
Xiaogang Feng ◽  
Zhihong Ye ◽  
Fred C. Lee ◽  
Dushan Borojevic
Keyword(s):  
Stability Margin ◽  
System Stability ◽  
Linear Control ◽  
Small Signal ◽  
System A ◽  
Novel Approach ◽  
Implementation Approach ◽  
Dc Bus ◽  
Response Current ◽  
Measurement Approach

PEBB (power electronics building block) systems are typical nonlinear systems. Under the conventional but still popular linear control design, the system stability margin varies from one operating point to another. This paper introduces a novel approach to monitoring the DC bus stability margin of a PEBB system online. At the steady state of the system, a small-signal perturbation current î p is injected into the DC bus, and the load-side response current î L is measured. By checking the validation |î L ( jw)| < |î p ( jw)|, the system stability margin can be examined. Experiments on a 48 V DC DPS demonstrate the proposed measurement approach. An implementation approach is also proposed for an 800 V DC PEBB-based testbed system.

Quantifying the Uncertainty of Energy Creation from Solar and Wind Farms in Different Locations

2016 ◽  
Vol 5 (2) ◽  
pp. 13-46 ◽  
Author(s):  
Roy Nersesian ◽  
Kenneth David Strang
Keyword(s):  
Large Scale ◽  
Wind Farm ◽  
Fossil Fuel ◽  
Wind Farms ◽  
System Stability ◽  
Novel Approach ◽  
Plant Operations ◽  
Generating Capacity ◽  
Demand Fluctuations ◽  
Battery Technology

This paper illustrates how to assess the risk associated with solar and wind farm energy creation by identifying the critical operational factors and then developing multivariate models. The study reveals that a dependence on solar and wind could place consumers at risk of interrupted service given the state of contemporary battery technology. Large scale electricity storage is not currently available which places a contingency risk on electricity generating capacity. More so, maintaining system stability where solar and wind play a significant role in generating electricity is a growing challenge facing utility operators. Therefore, the authors demonstrate how to build a model that quantifies uncertainty by matching uncontrollable supply to uncontrollable demand where a gravity battery may be installed as a buffer. This novel approach generalizes to fossil fuel and nuclear plant operations because demand fluctuations could be managed by storing surplus energy into a gravity battery to meet high peak periods.

scholarly journals Multibody dynamic modelling of a direct wind turbine drive train

2019 ◽  
Vol 44 (5) ◽  
pp. 519-547
Author(s):  
Saeed Asadi ◽  
Håkan Johansson
Keyword(s):  
Wind Turbine ◽  
Measurement Data ◽  
Dynamic Modelling ◽  
Simulation Software ◽  
Operating Conditions ◽  
Drive Train ◽  
Operational Experience ◽  
Main Bearing ◽  
Wide Range ◽  
Turbine Drive

Wind turbines normally have a long operational lifetime and experience a wide range of operating conditions. A representative set of these conditions is considered as part of a design process, as codified in standards. However, operational experience shows that failures occur more frequently than expected, the costlier of these including failures in the main bearings and gearbox. As modern turbines are equipped with sophisticated online systems, an important task is to evaluate the drive train dynamics from online measurement data. In particular, internal forces leading to fatigue can only be determined indirectly from other locations’ sensors. In this contribution, a direct wind turbine drive train is modelled using the floating frame of reference formulation for a flexible multibody dynamics system. The purpose is to evaluate drive train response based on blade root forces and bedplate motions. The dynamic response is evaluated in terms of main shaft deformation and main bearing forces under different wind conditions. The model was found to correspond well to a commercial wind turbine system simulation software (ViDyn).

Low Pressure Steam Turbine Exhaust Flow: Part 1 — CFD Coupling of LP Turbine and Condenser Neck

2016 ◽  
Author(s):  
Peter Stein ◽  
Dirk Telschow ◽  
Frederic Lamarque ◽  
Nuncio Colitto
Keyword(s):  
Steam Turbine ◽  
Measurement Data ◽  
Low Pressure ◽  
Computational Time ◽  
Feed Water ◽  
Cfd Modelling ◽  
Design Changes ◽  
Plant Level ◽  
Lp Turbine ◽  
Modelling Approach

Since many years the diffuser and exhaust of low pressure (LP) turbines have been in the focus of turbine development and accordingly broadly discussed within the scientific community. The pressure recovery gained within the diffuser significantly contributes to the turbine performance and therefore plenty of care is taken in investigations of the flow as well as optimization within this part of the turbine. However on a plant level the component following the LP turbine is the condenser, which is connected by the condenser neck. Typically the condenser neck is not fully designed to provide additional enthalpy recovery. Due to plant arrangement reasons, often it is full of built-ins like stiffening struts, feed-water heaters, extraction pipes, steam dump devices and others. It is vital to minimize the pressure losses across the condenser neck, in order to keep performance benefit, previously gained within the diffuser. As a general rule, each mbar of total pressure loss in a condenser neck may reduce the gross power output up to 0.1%. While turbines usually follow a modular approach, the condenser is typically designed plant specific. Therefore, on a plant level it is crucial to identify and evaluate the loss contributors and develop processes and tools which allow an accurate and efficient design process for an optimized condenser neck design. This needs to be performed as a coupled modelling approach, as both, turbine and condenser flow interact with each other. 3-D CFD tools enable a deep insight into the flow field and help to locally optimize the design, as they help to identify local losses and this even for small geometrical design changes. Unfortunately these tools are costly with respect to computational time and resources, if they are used to analyze a full condenser neck with all built-ins. Here 1-D modelling approaches can help to close the gap, as they can provide fast feedback, e.g. in a project tender phase, or can allow to quickly analyze design changes. For this they need a proper calibration and validation. This publication discusses the CFD modelling of a LP steam turbine coupled to a condenser neck and the validity of such calculations against measurement data. In the second publication (Part 2) a simplification of the gained information to a 1-D modelling approach will be discussed.

Sign in / Sign up

Export Citation Format

Share Document