Comparison of Multiple Reinforcement Learning and Deep Reinforcement Learning Methods for the Task Aimed at Achieving the Goal

Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) methods are a promising approach to solving complex tasks in the real world with physical robots. In this paper, we compare several reinforcement learning (Q-Learning, SARSA) and deep reinforcement learning (Deep Q-Network, Deep Sarsa) methods for a task aimed at achieving a specific goal using robotics arm UR3. The main optimization problem of this experiment is to find the best solution for each RL/DRL scenario and minimize the Euclidean distance accuracy error and smooth the resulting path by the Bézier spline method. The simulation and real word applications are controlled by the Robot Operating System (ROS). The learning environment is implemented using the OpenAI Gym library which uses the RVIZ simulation tool and the Gazebo 3D modeling tool for dynamics and kinematics.

Spline Curves Formation Given Extreme Derivatives

This paper is dedicated to development of mathematical models for polynomial spline curve formation given extreme vector derivatives. This theoretical problem is raised in the view of a wide variety of theoretical and practical problems considering motion of physical objects along certain trajectories with predetermined laws of variation of speed, acceleration, jerk, etc. The analysis of the existing body of work on computational geometry performed by the authors did not reveal any systematic research in mathematical model development dedicated to solution of similar tasks. The established purpose of the research is therefore to develop mathematical models of formation of spline curves based on polynomials of various orders modeling the determined trajectories. The paper presents mathematical models of spline curve formation given extreme derivatives of the initial orders. The paper considers construction of Hermite and Bézier spline curves of various orders consisting of various segments. The acquired mathematical models are generalized for the cases of vector derivatives of higher orders. The presented models are of systematic nature and are universal, i.e., they can be applied in formation of any polynomial spline curves given extreme vector derivatives. The paper provides a number of examples validating the presented models.

A new curvature-controlled stacking-line method for optimization design of compressor cascade considering surface smoothness

The paper presents an advanced parametric method of blade stacking lines in terms of sweep and lean based on controlled curvature. To the knowledge of the authors, there is no related approach reported in open literature that uses Bezier spline as the radial curvature distribution to improve the smoothness of the blade surface; most previous studies ignored the discontinuous slopes of curvature of the parametric curves. The parametric method called curvature-controlled stacking-line method (CCSLM) is performed by changing the magnitude of the sweep or lean. A fourth Bezier spline is adopted to define the curvature of spanwise stacking line directly ensuring surface smoothness. Then, the redesign cascades are created by sectional profiles stacked along the radial stacking lines which are obtained by twice integrating the Bezier spline. Then, the advanced method is conducted to optimize a high-subsonic controlled diffusion airfoil at design point, where the blade shape is generated in terms of lean. A single-objective optimization is performed using Kriging model and genetic algorithm to optimize total pressure loss, and the optimized geometry is obtained. The optimization results show that the blade design CCSLM has significant effects on the endwall flow vortex as well as radial loading distribution. The reduction of total pressure loss and secondary flow is also observed, and the aerodynamic performance is well improved compared with the original cascade.

A Trajectory Planning Based Controller to Regulate an Uncertain 3D Overhead Crane System

We introduce a control strategy to solve the regulation control problem, from the perspective of trajectory planning, for an uncertain 3D overhead crane. The proposed solution was developed based on an adaptive control approach that takes advantage of the passivity properties found in this kind of systems. We use a trajectory planning approach to preserve the accelerations and velocities inside of realistic ranges, to maintaining the payload movements as close as possible to the origin. To this end, we carefully chose a suitable S-curve based on the Bezier spline, which allows us to efficiently handle the load translation problem, considerably reducing the load oscillations. To perform the convergence analysis, we applied the traditional Lyapunov theory, together with Barbalat’s lemma. We assess the effectiveness of our control strategy with convincing numerical simulations.

Interpolating Industrial Robot Orientation with Hermite Spline Curve Based on Logarithmic Quaternion

Smooth orientation planning has an important influence on the working quality and service life as for industrial robot. Based on the logarithmic quaternion, a compact method to map a spline curve from Cartesian space to quaternion space is proposed, and consequently the multi-orientation smooth interpolation of quaternion is realized. Combining with the relevant example case, the detailed method and steps of multi-orientation interpolation are introduced for mapping Hermite spline curve into quaternion space, and the validity of the principle is verified by using the example case. The present multi-orientation smooth interpolation of quaternion has the characteristics of simple construction, easy implementation and intuitive understanding. The method is not only applicable to multi-orientation interpolation of quaternion with Hermite spline curve, but also can extended to the spline curves such as Bezier spline and B-spline.