Two efficient modifications of AZPRP conjugate gradient method with sufficient descent property

The conjugate gradient method can be applied in many fields, such as neural networks, image restoration, machine learning, deep learning, and many others. Polak–Ribiere–Polyak and Hestenses–Stiefel conjugate gradient methods are considered as the most efficient methods to solve nonlinear optimization problems. However, both methods cannot satisfy the descent property or global convergence property for general nonlinear functions. In this paper, we present two new modifications of the PRP method with restart conditions. The proposed conjugate gradient methods satisfy the global convergence property and descent property for general nonlinear functions. The numerical results show that the new modifications are more efficient than recent CG methods in terms of number of iterations, number of function evaluations, number of gradient evaluations, and CPU time.

Online Relaxation Refinement for Satisficing Planning: On Partial Delete Relaxation, Complete Hill-Climbing, and Novelty Pruning

In classical AI planning, heuristic functions typically base their estimates on a relaxation of the input task. Such relaxations can be more or less precise, and many heuristic functions have a refinement procedure that can be iteratively applied until the desired degree of precision is reached. Traditionally, such refinement is performed offline to instantiate the heuristic for the search. However, a natural idea is to perform such refinement online instead, in situations where the heuristic is not sufficiently accurate. We introduce several online-refinement search algorithms, based on hill-climbing and greedy best-first search. Our hill-climbing algorithms perform a bounded lookahead, proceeding to a state with lower heuristic value than the root state of the lookahead if such a state exists, or refining the heuristic otherwise to remove such a local minimum from the search space surface. These algorithms are complete if the refinement procedure satisfies a suitable convergence property. We transfer the idea of bounded lookaheads to greedy best-first search with a lightweight lookahead after each expansion, serving both as a method to boost search progress and to detect when the heuristic is inaccurate, identifying an opportunity for online refinement. We evaluate our algorithms with the partial delete relaxation heuristic hCFF, which can be refined by treating additional conjunctions of facts as atomic, and whose refinement operation satisfies the convergence property required for completeness. On both the IPC domains as well as on the recently published Autoscale benchmarks, our online-refinement search algorithms significantly beat state-of-the-art satisficing planners, and are competitive even with complex portfolios.

On three-dimensional SPH modelling of large-scale landslides

Lagrangian particle-based smoothed particle hydrodynamics (SPH) is increasingly widely used in landslide modelling. This paper investigates four important issues not addressed by previous studies on SPH modelling of large-scale landslides, i.e., convergence property, influence of constitutive parameters, scale effect and friction reduction, and influence of different treatments of the viscous effect. The GPU-acceleration technique is employed to achieve high-resolution three-dimensional (3D) modelling. The Baige landslide is investigated by comparing numerical results with field data, and detailed analyses on the four issues are provided. Suggestions on particle resolution, constitutive parameter, and formulations of viscous discretization are also presented for future SPH modelling of large-scale landslides.

Deep Reinforcement Learning-Based Accurate Control of Planetary Soft Landing

Planetary soft landing has been studied extensively due to its promising application prospects. In this paper, a soft landing control algorithm based on deep reinforcement learning (DRL) with good convergence property is proposed. First, the soft landing problem of the powered descent phase is formulated and the theoretical basis of Reinforcement Learning (RL) used in this paper is introduced. Second, to make it easier to converge, a reward function is designed to include process rewards like velocity tracking reward, solving the problem of sparse reward. Then, by including the fuel consumption penalty and constraints violation penalty, the lander can learn to achieve velocity tracking goal while saving fuel and keeping attitude angle within safe ranges. Then, simulations of training are carried out under the frameworks of Deep deterministic policy gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor Critic (SAC), respectively, which are of the classical RL frameworks, and all converged. Finally, the trained policy is deployed into velocity tracking and soft landing experiments, results of which demonstrate the validity of the algorithm proposed.

New scaled algorithm for non-linear conjugate gradients in unconstrained optimization

<p>A new scaled conjugate gradient (SCG) method is proposed throughout this paper, the SCG technique may be a special important generalization conjugate gradient (CG) method, and it is an efficient numerical method for solving nonlinear large scale unconstrained optimization. As a result, we proposed the new SCG method with a strong Wolfe condition (SWC) line search is proposed. The proposed technique's descent property, as well as its global convergence property, are satisfied without the use of any line searches under some suitable assumptions. The proposed technique's efficiency and feasibility are backed up by numerical experiments comparing them to traditional CG techniques.</p>

Hindsight Trust Region Policy Optimization

Reinforcement Learning (RL) with sparse rewards is a major challenge. We pro- pose Hindsight Trust Region Policy Optimization (HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with hindsight to tackle the challenge of sparse rewards. Hindsight refers to the algorithm’s ability to learn from information across goals, including past goals not intended for the current task. We derive the hindsight form of TRPO, together with QKL, a quadratic approximation to the KL divergence constraint on the trust region. QKL reduces variance in KL divergence estimation and improves stability in policy updates. We show that HTRPO has similar convergence property as TRPO. We also present Hindsight Goal Filtering (HGF), which further improves the learning performance for suitable tasks. HTRPO has been evaluated on various sparse-reward tasks, including Atari games and simulated robot control. Experimental results show that HTRPO consistently outperforms TRPO, as well as HPG, a state-of-the-art policy 14 gradient algorithm for RL with sparse rewards.

Brainless Walking: Animal Gaits Emerge From an Actuator Characteristic

In this study, we discovered a phenomenon in which a quadruped robot without any sensors or microprocessor can autonomously generate the various gait patterns of animals using actuator characteristics and select the gaits according to the speed. The robot has one DC motor on each limb and a slider-crank mechanism connected to the motor shaft. Since each motor is directly connected to a power supply, the robot only moves its foot on an elliptical trajectory under a constant voltage. Although this robot does not have any computational equipment such as sensors or microprocessors, when we applied a voltage to the motor, each limb begins to adjust its gait autonomously and finally converged to a steady gait pattern. Furthermore, by raising the input voltage from the power supply, the gait changed from a pace to a half-bound, according to the speed, and also we observed various gait patterns, such as a bound or a rotary gallop. We investigated the convergence property of the gaits for several initial states and input voltages and have described detailed experimental results of each gait observed.

Uniqueness and concentration for a fractional Kirchhoff problem with strong singularity

In this paper, we consider the following fractional Kirchhoff problem with strong singularity: $$ \textstyle\begin{cases} (1+ b\int _{\mathbb{R}^{3}}\int _{\mathbb{R}^{3}} \frac{ \vert u(x)-u(y) \vert ^{2}}{ \vert x-y \vert ^{3+2s}}\,\mathrm{d}x \,\mathrm{d}y )(-\Delta )^{s} u+V(x)u = f(x)u^{-\gamma }, & x \in \mathbb{R}^{3}, \\ u>0,& x\in \mathbb{R}^{3}, \end{cases} $$ { ( 1 + b ∫ R 3 ∫ R 3 | u ( x ) − u ( y ) | 2 | x − y | 3 + 2 s d x d y ) ( − Δ ) s u + V ( x ) u = f ( x ) u − γ , x ∈ R 3 , u > 0 , x ∈ R 3 , where $(-\Delta )^{s}$ ( − Δ ) s is the fractional Laplacian with $0< s<1$ 0 < s < 1 , $b>0$ b > 0 is a constant, and $\gamma >1$ γ > 1 . Since $\gamma >1$ γ > 1 , the energy functional is not well defined on the work space, which is quite different with the situation of $0<\gamma <1$ 0 < γ < 1 and can lead to some new difficulties. Under certain assumptions on V and f, we show the existence and uniqueness of a positive solution $u_{b}$ u b by using variational methods and the Nehari manifold method. We also give a convergence property of $u_{b}$ u b as $b\rightarrow 0$ b → 0 , where b is regarded as a positive parameter.

A note on H-convergence

In this paper we study the H-convergence property for the uniformly bounded sequences of square matrices $\left\{ A_{\varepsilon} \in L^{\infty} (D; \mathbb{R}^{n \times n}) \right\}_{\varepsilon > 0}$. We derive the sufficient conditions, which guarantee the coincidence of $H$-limit with the weak-* limit of such sequences in $L^{\infty} (D; \mathbb{R}^{n \times n})$.

Show off the efficiency of dai-liao method in merging technology for monotonous non-linear problems

<span>In this article, we give a new modification for the Dai-Liao method to solve monotonous nonlinear problems. In our modification, we relied on two important procedures, one of them was the projection method and the second was the method of damping the quasi-Newton condition. The new approach of derivation yields two new parameters for the conjugated gradient direction which, through some conditions, we have demonstrated the sufficient descent property for them. Under some necessary conditions, the new approach achieved global convergence property. Numerical results show how efficient the new approach is when compared with basic similar classic methods.</span>