In this paper, we consider the artificial neural networks for solving
the differential equation with boundary layer, in which the gradient of
the solution changes sharply near the boundary layer. The solution of
the boundary layer problems poses a huge challenge to both traditional
numerical methods and artificial neural network methods. By theoretical
analyzing the changing rate of the weights of first hidden layer near
the boundary layer, a mapping strategy is added in traditional neural
network to improve the convergence of the loss function. Numerical
examples are carried out for the 1D and 2D convection-diffusion equation
with boundary layer. The results demonstrate that the modified neural
networks significantly improve the ability in approximating the
solutions with sharp gradient.
Adaptive Weight Decay for Deep Neural Networks
