Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning

In recent years, tremendous progress has been made on numerical algorithms for solving partial differential equations (PDEs) in a very high dimension, using ideas from either nonlinear (multilevel) Monte Carlo or deep learning. They are potentially free of the curse of dimensionality for many different applications and have been proven to be so in the case of some nonlinear Monte Carlo methods for nonlinear parabolic PDEs. In this paper, we review these numerical and theoretical advances. In addition to algorithms based on stochastic reformulations of the original problem, such as the multilevel Picard iteration and the deep backward stochastic differential equations method, we also discuss algorithms based on the more traditional Ritz, Galerkin, and least square formulations. We hope to demonstrate to the reader that studying PDEs as well as control and variational problems in very high dimensions might very well be among the most promising new directions in mathematics and scientific computing in the near future.

On numerical solution of nonlinear parabolic multicomponent diffusion-reaction problems

This work continues our previous analysis concerning the numerical solution of the multi-component mass transfer equations. The present test problems are two-dimensional, parabolic, non-linear, diffusion- reaction equations. An implicit finite difference method was used to discretize the mathematical model equations. The algorithm used to solve the non-linear system resulted for each time step is the modified Picard iteration. The numerical performances of the preconditioned conjugate gradient algorithms (BICGSTAB and GMRES) in solving the linear systems of the modified Picard iteration were analysed in detail. The numerical results obtained show good numerical performances.

Numerical Solution of Two-Dimensional Fredholm–Volterra Integral Equations of the Second Kind

The paper presents an iterative numerical method for approximating solutions of two-dimensional Fredholm–Volterra integral equations of the second kind. As these equations arise in many applications, there is a constant need for accurate, but fast and simple to use numerical approximations to their solutions. The method proposed here uses successive approximations of the Mann type and a suitable cubature formula. Mann’s procedure is known to converge faster than the classical Picard iteration given by the contraction principle, thus yielding a better numerical method. The existence and uniqueness of the solution is derived under certain conditions. The convergence of the method is proved, and error estimates for the approximations obtained are given. At the end, several numerical examples are analyzed, showing the applicability of the proposed method and good approximation results. In the last section, concluding remarks and future research ideas are discussed.

An Improved Picard Iteration Scheme for Simulating Unsaturated Flow in Layered Porous Media

Picard iteration method is commonly used to obtain numerical solution of unsaturated flow in porous media. However, because the system of linear equations derived from Richards equation is seriously ill-conditioned, Picard iteration has slow convergence rate and low computational efficiency, particularly in layered porous media. In this study, control volume method based on non-uniform nodes is used to discrete Richards equation. To improve the convergence rate of Picard iteration, we combine the non-uniform multigrid correction method with the multistep preprocessing technology. Thus, an improved Picard iteration scheme with multistep preconditioner based on non-uniform multigrid correction method (NMG-MPPI(m)) is proposed to model 1D unsaturated flow in layered porous media. Three test cases were used to verify the proposed schemes. The result shows that the condition number of the coefficient matrix has been greatly reduced using the multistep preconditioner. Numerical results indicate that NMG-MPPI(m) can solve Richards equation at a faster convergence rate, with higher calculation accuracy and good robustness. Compared with conventional Picard iteration, NMG-MPPI(m) shows a very high speed-up ratio. As a result, the improved Picard iteration scheme has good application for simulating unsaturated flow in layered porous media.

Modelling Unsaturated Flow in Porous Media Using an Improved Picard Iteration Scheme

The numerical solution of various systems of linear equations describing fluid infiltration uses the Picard iteration (PI). However, because many such systems are ill-conditioned, the solution process often has a poor convergence rate, making it very time-consuming. In this study, a control volume method based on non-uniform nodes is used to discretize the Richards equation, and adaptive relaxation is combined with a multistep preconditioner to improve the convergence rate of PI. The resulting adaptive relaxed PI with multistep preconditioner (MP(m)-ARPI) is used to simulate unsaturated flow in porous media. Three examples are used to verify the proposed schemes. The results show that MP(m)-ARPI can effectively reduce the condition number of the coefficient matrix for the system of linear equations. Compared with conventional PI, MP(m)-ARPI achieves faster convergence, higher computational efficiency, and enhanced robustness. These results demonstrate that improved scheme is an excellent prospect for simulating unsaturated flow in porous media.

An attitude updating algorithm using Picard iteration and higher degree polynomial

To accurately track body attitude under high dynamic environments, a new attitude updating algorithm for the strapdown inertial navigation system is proposed after further applying higher degree polynomial to the quaternion Picard iteration (QPI) algorithm. With QPI, calculation error introduced by Picard approximation can be eliminated, but the angular rate fitting error introduced by substituting polynomial for angular rate of body will still affect the accuracy of the attitude updating algorithms which are designed based on polynomial model. Hence, a five- rather three-degree polynomial constructing method using four samples of gyro outputs with coning motion constrain is designed and tested. Simulation results indicate the proposed method owns more accuracy than QPI, optimal coning algorithm, and Fourth4Rot under both low and high dynamic environments.

On the robust Newton’s method with the Mann iteration and the artistic patterns from its dynamics

There are two main aims of this paper. The first one is to show some improvement of the robust Newton’s method (RNM) introduced recently by Kalantari. The RNM is a generalisation of the well-known Newton’s root finding method. Since the base method is undefined at critical points, the RNM allows working also at such points. In this paper, we improve the RNM method by applying the Mann iteration instead of the standard Picard iteration. This leads to an essential decrease in the number of root finding steps without visible destroying the sharp boundaries among the basins of attractions presented in polynomiographs. Furthermore, we investigate visually the dynamics of the RNM with the Mann iteration together with the basins of attraction for varying Mann’s iteration parameter with the help of polynomiographs for several polynomials. The second aim of this paper is to present the intriguing polynomiographs obtained from the dynamics of the RNM with the Mann iteration under various sequences used in this iteration. The obtained polynomiographs differ considerably from the ones obtained with the RNM and are interesting from the artistic perspective. Moreover, they can easily find applications in wallpaper or fabric design.