Reinforcement Learning based on Deep Deterministic Policy Gradient for Roll Control of Underwater Vehicle

The existing underwater vehicle controller design is applied by linearizing the nonlinear dynamics model to a specific motion section. Since the linear controller has unstable control performance in a transient state, various studies have been conducted to overcome this problem. Recently, there have been studies to improve the control performance in the transient state by using reinforcement learning. Reinforcement learning can be largely divided into value-based reinforcement learning and policy-based reinforcement learning. In this paper, we propose the roll controller of underwater vehicle based on Deep Deterministic Policy Gradient(DDPG) that learns the control policy and can show stable control performance in various situations and environments. The performance of the proposed DDPG based roll controller was verified through simulation and compared with the existing PID and DQN with Normalized Advantage Functions based roll controllers.

A Deep Neural Network-based Nonlinear Dynamics Model for the Prediction of Lateral-longitudinal Vehicle Dynamics

The vehicle multibody model can be used for accurate coupling dynamics, but it has higher computational complexity. Numerical stability during integration is also very challenging, especially in complicated driving situations. This issue can be substantially alleviated by using a data-driven nonlinear dynamics model owing to its computational speed and robust generalization. In this work, we propose a deep neural network (DNN)-based modeling approach for predicting lateral-longitudinal vehicle dynamics. Dynamic simulations of vehicle systems are performed based on a semirecursive multibody formulation for data acquisition. The data are then used for training and testing the DNN model. The DNN inputs are the torque applied on wheels and the initial vehicle speed that imitates a double lane change maneuver with acceleration and deceleration. The DNN outputs are the longitudinal driving distance, lateral driving distance, final longitudinal velocities, final lateral velocities, and yaw angle. The dynamic responses obtained from the DNN model are compared with the multibody results. Furthermore, the accuracy of the DNN model is investigated in terms of error functions. The DNN model is finally verified via the results of a commercial software package. The results show that the DNN vehicle dynamics model predicts accurate dynamic responses in real time. The DNN model can be used for real-time simulation and preview control in autonomous vehicles.

Double-Hoist Dynamics and Oscillation Control of a Quadcopter Carrying a Swinging and Twisting Load

Quadcopters can serve as aerial cranes that are able to suspend large-size loads below the fuselage for material-handling services. Double-hoist mechanisms help quadcopter to transport bulky loads effectively. However, double-hoist mechanisms exhibit strong coupling effect among the quadcopter’s attitude, load swing, and load twisting. Different from the single-hoist mechanisms, no effects have been directed at quadcopter slung loads with double-hoist mechanisms. A novel nonlinear dynamics model of a quadcopter carrying a distributed-mass load by double-hoist mechanisms is given in this paper. This explicit model captures the coupling between the quadcopter and load motion. Then a novel control method is proposed to reduce the swing and twisting of the load simultaneously. The simulation results illustrate that the control method succeeds in limiting the unwanted oscillations of quadcopter attitudes, load swing and twisting.

Universal Adaptive Neural Network Predictive Algorithm for Remotely Piloted Unmanned Combat Aerial Vehicle in Wireless Sensor Network

Remotely piloted unmanned combat aerial vehicle (UCAV) will be a prospective mode of air fight in the future, which can remove the physical restraint of the pilot, maximize the performance of the fighter and effectively reduce casualties. However, it has two difficulties in this mode: (1) There is greater time delay in the network of pilot-wireless sensor-UCAV, which can degrade the piloting performance. (2) Designing of a universal predictive method is very important to pilot different UCAVs remotely, even if the model of the control augmentation system of the UCAV is totally unknown. Considering these two issues, this paper proposes a novel universal modeling method, and establishes a universal nonlinear uncertain model which uses the pilot’s remotely piloted command as input and the states of the UCAV with a control augmentation system as output. To deal with the nonlinear uncertainty of the model, a neural network observer is proposed to identify the nonlinear dynamics model online. Meanwhile, to guarantee the stability of the overall observer system, an adaptive law is designed to adjust the neural network weights. To solve the greater transmission time delay existing in the pilot-wireless sensor-UCAV closed-loop system, a time-varying delay state predictor is designed based on the identified nonlinear dynamics model to predict the time delay states. Moreover, the overall observer-predictor system is proved to be uniformly ultimately bounded (UUB). Finally, two simulations verify the effectiveness and universality of the proposed method. The results indicate that the proposed method has desirable performance of accurately compensating the time delay and has universality of remotely piloting two different UCAVs.

Integrated Vibration Control of In-Wheel Motor-Suspensions Coupling System via Dynamics Parameter Optimization

In order to integrally control the vibration of in-wheel motor- (IWM-) suspensions coupling system of an electric vehicle, a novel nonlinear dynamics model of the coupling system, which consists of the motor magnetic gap (MMG), is established. Synthesizing subtargets of the vertical vibration acceleration of bodywork, the vertical deformation of tire, the suspension travel, and the vertical fluctuation of MMG, a composite optimization mathematical model is set up. Based on artificial fish swarm algorithm (AFSA), a novel dynamics parameter optimization method is proposed to search the optimal parameter combination existing in the nonlinear dynamics model. Simulation analyses demonstrate that the proposed optimization method is superior to genetic algorithm (GA) under the same optimization conditions, and it can significantly decrease the fluctuation of MMG and improve ride comfort.

Nonlinear Modeling and Flight Validation of a Small-Scale Compound Helicopter

The compound configuration is a good option for helicopters to break through speed limitation and improve maneuverability. However, the compound configuration applied on the small-scale helicopter has not been investigated in detail. In this study, an 11-state nonlinear dynamics model of a small-scale compound helicopter was established with the help of first physical principles and linear modification method. The ducted fan, free-rotate wing and horizontal stabilizer were considered in the compound configurations. To validate the accuracy of the model, high-quality flight data were obtained in hover and forward flights from 15 m/s to 32 m/s. Results show that the overall responses of the developed nonlinear model matched the hover data. In forward flight, it was proved that the nonlinear model has high accuracy in agreement with trim results and time-domain simulations. The wing model works well below 27 m/s. Furthermore, the effectiveness of the elevator and aileron in high speed was also verified in the simulation of a coordinated turn.