Introducing a training methodology for cellular neural networks solving partial differential equations

2009 ◽  
Author(s):  
M. J. Aein ◽  
H.A. Talebi
Keyword(s):  
Neural Networks ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Cellular Neural Networks ◽  
Partial Differential ◽  
Training Methodology

A Novel General and Robust Method Based on NAOP for Solving Nonlinear Ordinary Differential Equations and Partial Differential Equations by Cellular Neural Networks

2013 ◽  
Vol 135 (3) ◽  
Author(s):  
Jean Chamberlain Chedjou ◽  
Kyandoghere Kyamakya
Keyword(s):  
Neural Networks ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Ordinary Differential Equations ◽  
Cellular Neural Networks ◽  
Nonlinear Ordinary Differential Equations ◽  
Van Der Pol ◽  
Partial Differential ◽  
Nonlinear Odes ◽  
First Order

This paper develops and validates through a series of presentable examples, a comprehensive high-precision, and ultrafast computing concept for solving nonlinear ordinary differential equations (ODEs) and partial differential equations (PDEs) with cellular neural networks (CNN). The core of this concept is a straightforward scheme that we call "nonlinear adaptive optimization (NAOP),” which is used for a precise template calculation for solving nonlinear ODEs and PDEs through CNN processors. One of the key contributions of this work is to demonstrate the possibility of transforming different types of nonlinearities displayed by various classical and well-known nonlinear equations (e.g., van der Pol-, Rayleigh-, Duffing-, Rössler-, Lorenz-, and Jerk-equations, just to name a few) unto first-order CNN elementary cells, and thereby enabling the easy derivation of corresponding CNN templates. Furthermore, in the case of PDE solving, the same concept also allows a mapping unto first-order CNN cells while considering one or even more nonlinear terms of the Taylor's series expansion generally used in the transformation of a PDE in a set of coupled nonlinear ODEs. Therefore, the concept of this paper does significantly contribute to the consolidation of CNN as a universal and ultrafast solver of nonlinear ODEs and/or PDEs. This clearly enables a CNN-based, real-time, ultraprecise, and low-cost computational engineering. As proof of concept, two examples of well-known ODEs are considered namely a second-order linear ODE and a second order nonlinear ODE of the van der Pol type. For each of these ODEs, the corresponding precise CNN templates are derived and are used to deduce the expected solutions. An implementation of the concept developed is possible even on embedded digital platforms (e.g., field programmable gate array (FPGA), digital signal processor (DSP), graphics processing unit (GPU), etc.). This opens a broad range of applications. Ongoing works (as outlook) are using NAOP for deriving precise templates for a selected set of practically interesting ODEs and PDEs equation models such as Lorenz-, Rössler-, Navier Stokes-, Schrödinger-, Maxwell-, etc.

Applying neural network for identification of land surface model parameters

2021 ◽  
Author(s):  
Ruslan Chernyshev ◽  
Mikhail Krinitskiy ◽  
Viktor Stepanenko
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Thermodynamic Model ◽  
Land Surface ◽  
Land Surface Model ◽  
Surface Model ◽  
Partial Differential

<p>This work is devoted to development of neural networks for identification of partial differential equations (PDE) solved in the land surface scheme of INM RAS Earth System model (ESM). Atmospheric and climate models are in the top of the most demanding for supercomputing resources among research applications. Spatial resolution and a multitude of physical parameterizations used in ESMs continuously increase. Most of parameters are still poorly constrained, many of them cannot be measured directly. To optimize model calibration time, using neural networks looks a promising approach. Neural networks are already in wide use in satellite imaginary (Su Jeong Lee, et al, 2015; Krinitskiy M. et al, 2018) and for calibrating parameters of land surface models (Yohei Sawada el al, 2019). Neural networks have demonstrated high efficiency in solving conventional problems of mathematical physics (Lucie P. Aarts el al, 2001; Raissi M. et al, 2020). </p><p>We develop a neural networks for optimizing parameters of nonlinear soil heat and moisture transport equation set. For developing we used Python3 based programming tools implemented on GPUs and Ascend platform, provided by Huawei. Because of using hybrid approach combining neural network and classical thermodynamic equations, the major purpose was finding the way to correctly calculate backpropagation gradient of error function, because model trains and is being validated on the same temperature data, while model output is heat equation parameter, which is typically not known. Neural network model has been runtime trained using reference thermodynamic model calculation with prescribed parameters, every next thermodynamic model step has been used for fitting the neural network until it reaches the loss function tolerance.</p><p>Literature:</p><p>1.     Aarts, L.P., van der Veer, P. “Neural Network Method for Solving Partial Differential Equations”. Neural Processing Letters 14, 261–271 (2001). https://doi.org/10.1023/A:1012784129883</p><p>2.     Raissi, M., P. Perdikaris and G. Karniadakis. “Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations.” ArXiv abs/1711.10561 (2017): n. pag.</p><p>3.     Lee, S.J., Ahn, MH. & Lee, Y. Application of an artificial neural network for a direct estimation of atmospheric instability from a next-generation imager. Adv. Atmos. Sci. 33, 221–232 (2016). https://doi.org/10.1007/s00376-015-5084-9</p><p>4.     Krinitskiy M, Verezemskaya P, Grashchenkov K, Tilinina N, Gulev S, Lazzara M. Deep Convolutional Neural Networks Capabilities for Binary Classification of Polar Mesocyclones in Satellite Mosaics. Atmosphere. 2018; 9(11):426.</p><p>5.     Sawada, Y.. “Machine learning accelerates parameter optimization and uncertainty assessment of a land surface model.” ArXiv abs/1909.04196 (2019): n. pag.</p><p>6.     Shufen Pan et al. Evaluation of global terrestrial evapotranspiration using state-of-the-art approaches in remote sensing, machine learning and land surface modeling. Hydrol. Earth Syst. Sci., 24, 1485–1509 (2020)</p><p>7.     Chaney, Nathaniel & Herman, Jonathan & Ek, M. & Wood, Eric. (2016). Deriving Global Parameter Estimates for the Noah Land Surface Model using FLUXNET and Machine Learning: Improving Noah LSM Parameters. Journal of Geophysical Research: Atmospheres. 121. 10.1002/2016JD024821.</p><p> </p><p> </p>

CONTINUOUS NEURAL NETWORKS APPLIED TO IDENTIFY A CLASS OF UNCERTAIN PARABOLIC PARTIAL DIFFERENTIAL EQUATIONS

2010 ◽  
Vol 01 (04) ◽  
pp. 485-508
Author(s):  
R. FUENTES ◽  
A. POZNYAK ◽  
I. CHAIREZ ◽  
M. FRANCO ◽  
T. POZNYAK
Keyword(s):  
Neural Networks ◽  
Mathematical Model ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Parametric Modeling ◽  
Parabolic Partial Differential Equations ◽  
Qualitative Behavior ◽  
Compact Set ◽  
Large Set ◽  
Partial Differential

There are a lot of examples in science and engineering that may be described using a set of partial differential equations (PDEs). Those PDEs are obtained applying a process of mathematical modeling using complex physical, chemical, etc. laws. Nevertheless, there are many sources of uncertainties around the aforementioned mathematical representation. It is well known that neural networks can approximate a large set of continuous functions defined on a compact set. If the continuous mathematical model is incomplete or partially known, the methodology based on Differential Neural Network (DNN) provides an effective tool to solve problems on control theory such as identification, state estimation, trajectories tracking, etc. In this paper, a strategy based on DNN for the no parametric identification of a mathematical model described by parabolic partial differential equations is proposed. The identification solution allows finding an exact expression for the weights' dynamics. The weights adaptive laws ensure the "practical stability" of DNN trajectories. To verify the qualitative behavior of the suggested methodology, a no parametric modeling problem for a couple of distributed parameter plants is analyzed: the plug-flow reactor model and the anaerobic digestion system. The results obtained in the numerical simulations confirm the identification capability of the suggested methodology.

Sign in / Sign up

Export Citation Format

Share Document