scholarly journals Massive computational acceleration by using neural networks to emulate mechanism-based biological models

2019 ◽  
Author(s):  
Shangying Wang ◽  
Kai Fan ◽  
Nan Luo ◽  
Yangxiaolu Cao ◽  
Feilun Wu ◽  
...  
Keyword(s):  
Neural Network ◽  
Differential Equation ◽  
Neural Networks ◽  
Differential Equations ◽  
System Dynamics ◽  
Mechanistic Model ◽  
Biological Models ◽  
Partial Differential ◽  
Parametric Space ◽  
Differential Equation Models

AbstractMechanism-based mathematical models are the foundation for diverse applications. It is often critical to explore the massive parametric space for each model. However, for many applications, such as agent-based models, partial differential equations, and stochastic differential equations, this practice can impose a prohibitive computational demand. To overcome this limitation, we present a fundamentally new framework to improve computational efficiency by orders of magnitude. The key concept is to train an artificial neural network using a limited number of simulations generated by a mechanistic model. This number is small enough such that the simulations can be completed in a short time frame but large enough to enable reliable training of the neural network. The trained neural network can then be used to explore the system dynamics of a much larger parametric space. We demonstrate this notion by training neural networks to predict self-organized pattern formation and stochastic gene expression. With this framework, we can predict not only the 1-D distribution in space (for partial differential equation models) and probability density function (for stochastic differential equation models) of variables of interest with high accuracy, but also novel system dynamics not present in the training sets. We further demonstrate that using an ensemble of neural networks enables the self-contained evaluation of the quality of each prediction. Our work can potentially be a platform for faster parametric space screening of biological models with user defined objectives.

scholarly journals Massive computational acceleration by using neural networks to emulate mechanism-based biological models

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Shangying Wang ◽  
Kai Fan ◽  
Nan Luo ◽  
Yangxiaolu Cao ◽  
Feilun Wu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Mechanistic Model ◽  
Time Frame ◽  
Biological Applications ◽  
Biological Models ◽  
Parametric Space ◽  
Trained Neural Network ◽  
Short Time

Abstract For many biological applications, exploration of the massive parametric space of a mechanism-based model can impose a prohibitive computational demand. To overcome this limitation, we present a framework to improve computational efficiency by orders of magnitude. The key concept is to train a neural network using a limited number of simulations generated by a mechanistic model. This number is small enough such that the simulations can be completed in a short time frame but large enough to enable reliable training. The trained neural network can then be used to explore a much larger parametric space. We demonstrate this notion by training neural networks to predict pattern formation and stochastic gene expression. We further demonstrate that using an ensemble of neural networks enables the self-contained evaluation of the quality of each prediction. Our work can be a platform for fast parametric space screening of biological models with user defined objectives.

Applying neural network for identification of land surface model parameters

2021 ◽  
Author(s):  
Ruslan Chernyshev ◽  
Mikhail Krinitskiy ◽  
Viktor Stepanenko
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Thermodynamic Model ◽  
Land Surface ◽  
Land Surface Model ◽  
Surface Model ◽  
Partial Differential

<p>This work is devoted to development of neural networks for identification of partial differential equations (PDE) solved in the land surface scheme of INM RAS Earth System model (ESM). Atmospheric and climate models are in the top of the most demanding for supercomputing resources among research applications. Spatial resolution and a multitude of physical parameterizations used in ESMs continuously increase. Most of parameters are still poorly constrained, many of them cannot be measured directly. To optimize model calibration time, using neural networks looks a promising approach. Neural networks are already in wide use in satellite imaginary (Su Jeong Lee, et al, 2015; Krinitskiy M. et al, 2018) and for calibrating parameters of land surface models (Yohei Sawada el al, 2019). Neural networks have demonstrated high efficiency in solving conventional problems of mathematical physics (Lucie P. Aarts el al, 2001; Raissi M. et al, 2020). </p><p>We develop a neural networks for optimizing parameters of nonlinear soil heat and moisture transport equation set. For developing we used Python3 based programming tools implemented on GPUs and Ascend platform, provided by Huawei. Because of using hybrid approach combining neural network and classical thermodynamic equations, the major purpose was finding the way to correctly calculate backpropagation gradient of error function, because model trains and is being validated on the same temperature data, while model output is heat equation parameter, which is typically not known. Neural network model has been runtime trained using reference thermodynamic model calculation with prescribed parameters, every next thermodynamic model step has been used for fitting the neural network until it reaches the loss function tolerance.</p><p>Literature:</p><p>1.     Aarts, L.P., van der Veer, P. “Neural Network Method for Solving Partial Differential Equations”. Neural Processing Letters 14, 261–271 (2001). https://doi.org/10.1023/A:1012784129883</p><p>2.     Raissi, M., P. Perdikaris and G. Karniadakis. “Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations.” ArXiv abs/1711.10561 (2017): n. pag.</p><p>3.     Lee, S.J., Ahn, MH. & Lee, Y. Application of an artificial neural network for a direct estimation of atmospheric instability from a next-generation imager. Adv. Atmos. Sci. 33, 221–232 (2016). https://doi.org/10.1007/s00376-015-5084-9</p><p>4.     Krinitskiy M, Verezemskaya P, Grashchenkov K, Tilinina N, Gulev S, Lazzara M. Deep Convolutional Neural Networks Capabilities for Binary Classification of Polar Mesocyclones in Satellite Mosaics. Atmosphere. 2018; 9(11):426.</p><p>5.     Sawada, Y.. “Machine learning accelerates parameter optimization and uncertainty assessment of a land surface model.” ArXiv abs/1909.04196 (2019): n. pag.</p><p>6.     Shufen Pan et al. Evaluation of global terrestrial evapotranspiration using state-of-the-art approaches in remote sensing, machine learning and land surface modeling. Hydrol. Earth Syst. Sci., 24, 1485–1509 (2020)</p><p>7.     Chaney, Nathaniel & Herman, Jonathan & Ek, M. & Wood, Eric. (2016). Deriving Global Parameter Estimates for the Noah Land Surface Model using FLUXNET and Machine Learning: Improving Noah LSM Parameters. Journal of Geophysical Research: Atmospheres. 121. 10.1002/2016JD024821.</p><p> </p><p> </p>

Adaptive Diversification and Speciation as Pattern Formation in Partial Differential Equation Models

2011 ◽  
Author(s):  
Michael Doebeli
Keyword(s):  
Differential Equation ◽  
Partial Differential Equation ◽  
Partial Differential Equations ◽  
Pattern Formation ◽  
Differential Equations ◽  
Adaptive Dynamics ◽  
Partial Differential ◽  
Adaptive Diversification ◽  
Differential Equation Models ◽  
Phenotype Distribution

This chapter discusses partial differential equation models. Partial differential equations can describe the dynamics of phenotype distributions of polymorphic populations, and they allow for a mathematically concise formulation from which some analytical insights can be obtained. It has been argued that because partial differential equations can describe polymorphic populations, results from such models are fundamentally different from those obtained using adaptive dynamics. In partial differential equation models, diversification manifests itself as pattern formation in phenotype distribution. More precisely, diversification occurs when phenotype distributions become multimodal, with the different modes corresponding to phenotypic clusters, or to species in sexual models. Such pattern formation occurs in partial differential equation models for competitive as well as for predator–prey interactions.

scholarly journals Modelling and control of flexible guided lifting system with output constraints and unknown input hysteresis

2019 ◽  
Vol 26 (1-2) ◽  
pp. 112-128
Author(s):  
Naige Wang ◽  
Guohua Cao ◽  
Lei Wang ◽  
Yan Lu ◽  
Zhencai Zhu
Keyword(s):  
Neural Network ◽  
Differential Equation ◽  
Differential Equations ◽  
Governing Equation ◽  
Control Input ◽  
Partial Differential ◽  
Adaptive Neural Network Control ◽  
Output Constraints ◽  
Lifting System ◽  
And Control

Modelling and control vibration is studied for the flexible guided lifting system in the presence of output constraints, input hysteresis, guided rope fault, etc. Flexible guided lifting system, subjected to external disturbances from the boundary disturbance or fluid interaction, is an inherent distributed parameter system with time-varying length and infinite dimensions. According to extended Hamilton’s principle, the governing equation in the form of hybrid partial differential equations and ordinary differential equations is derived to reflect the dynamic response of such multiple ropes under the boundary disturbances and multiple constraints. Adaptive neural network control combining with backstepping technique is subsequently designed to suppress undesirable vibration and stabilise the system, where the neural network is provided as a feedforward compensator for the unknown hysteresis nonlinearities in the control input. Asymptotic stability and uniform boundedness of the system are guaranteed through the Lyapunov theorem, which indicates that all states are uniformly convergent by LaSalle’s invariance principle. The original governing equation is solved numerically by using the finite difference method, where simulation results are illustrated to validate the efficiency of the hybrid partial differential equation and ordinary differential equation model and the adaptive stabilisation design.

scholarly journals Nonindependent Session Recommendation Based on Ordinary Differential Equation

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Zhenyu Yang ◽  
Mingge Zhang ◽  
Guojing Liu ◽  
Mingyu Li
Keyword(s):  
Neural Network ◽  
Differential Equation ◽  
Neural Networks ◽  
Ordinary Differential Equation ◽  
Differential Equations ◽  
Ordinary Differential Equations ◽  
Deep Neural Networks ◽  
Semantic Information ◽  
User Behavior ◽  
User Behaviors

The recommendation method based on user sessions is mainly to model sessions as sequences in the assumption that user behaviors are independent and identically distributed, and then to use deep semantic information mining through Deep Neural Networks. Nevertheless, user behaviors may be a nonindependent intention at irregular points in time. For example, users may buy painkillers, books, or clothes for different reasons at different times. However, this has not been taken seriously in previous studies. Therefore, we propose a session recommendation method based on Neural Differential Equations in an attempt to predict user behavior forward or backward from any point in time. We used Ordinary Differential Equations to train the Graph Neural Network and could predict forward or backward at any point in time to model the user's nonindependent sessions. We tested for four real datasets and found that our model achieved the expected results and was superior to the existing session-based recommendations.

GIẢI PHƯƠNG TRÌNH ĐẠO HÀM RIÊNG SỬ DỤNG MẠNG NEURAL NHÂN TẠO

2020 ◽  
Vol 45 (03) ◽  
Author(s):  
HO DAC QUAN ◽  
HUYNH TRUNG HIEU
Keyword(s):  
Neural Network ◽  
Differential Equation ◽  
Neural Networks ◽  
Partial Differential Equation ◽  
Partial Differential ◽  
Neural Network Method ◽  
Network Method ◽  
Hidden Layer

Phương trình đạo hàm riêng đã được ứng dụng rộng rãi trong các lĩnh vực khác nhau của đời sống như vật lý, hóa học, kinh tế, xử lý ảnh vv. Trong bài báo này chúng tôi trình bày một phương pháp giải phương trình đạo hàm riêng (partial differential equation - PDE) thoả điều kiện biên Dirichlete sửdụng mạng neural truyền thẳng một lớp ẩn (single-hidden layer feedfordward neural networks - SLFN) gọi là phương pháp mạng neural (neural network method – NNM). Các tham số của mạng neural được xác định dựa trên thuật toán huấn luyện mạng lan truyền ngược (backpropagation - BP). Kết quả nghiệm PDE thu được bằng phương pháp NNM chính xác hơn so với nghiệm PDE giải bằng phương pháp sai phân hữu hạn.

scholarly journals Efficient Methods for Parameter Estimation of Ordinary and Partial Differential Equation Models of Viral Hepatitis Kinetics

Mathematics ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. 1483
Author(s):  
Alexander Churkin ◽  
Stephanie Lewkiewicz ◽  
Vladimir Reinharz ◽  
Harel Dahari ◽  
Danny Barash
Keyword(s):  
Differential Equation ◽  
Partial Differential Equation ◽  
Parameter Estimation ◽  
Differential Equations ◽  
Viral Hepatitis ◽  
Antiviral Treatment ◽  
Boundary Values ◽  
Partial Differential ◽  
Differential Equation Models ◽  
Age Structured

Parameter estimation in mathematical models that are based on differential equations is known to be of fundamental importance. For sophisticated models such as age-structured models that simulate biological agents, parameter estimation that addresses all cases of data points available presents a formidable challenge and efficiency considerations need to be employed in order for the method to become practical. In the case of age-structured models of viral hepatitis dynamics under antiviral treatment that deal with partial differential equations, a fully numerical parameter estimation method was developed that does not require an analytical approximation of the solution to the multiscale model equations, avoiding the necessity to derive the long-term approximation for each model. However, the method is considerably slow because of precision problems in estimating derivatives with respect to the parameters near their boundary values, making it almost impractical for general use. In order to overcome this limitation, two steps have been taken that significantly reduce the running time by orders of magnitude and thereby lead to a practical method. First, constrained optimization is used, letting the user add constraints relating to the boundary values of each parameter before the method is executed. Second, optimization is performed by derivative-free methods, eliminating the need to evaluate expensive numerical derivative approximations. The newly efficient methods that were developed as a result of the above approach are described for hepatitis C virus kinetic models during antiviral therapy. Illustrations are provided using a user-friendly simulator that incorporates the efficient methods for both the ordinary and partial differential equation models.

scholarly journals Partial differential equation models in macroeconomics

2014 ◽  
Vol 372 (2028) ◽  
pp. 20130397 ◽  
Author(s):  
Yves Achdou ◽  
Francisco J. Buera ◽  
Jean-Michel Lasry ◽  
Pierre-Louis Lions ◽  
Benjamin Moll
Keyword(s):  
Differential Equation ◽  
Partial Differential Equation ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Future Research ◽  
Partial Differential ◽  
Open Questions ◽  
Differential Equation Models

The purpose of this article is to get mathematicians interested in studying a number of partial differential equations (PDEs) that naturally arise in macroeconomics. These PDEs come from models designed to study some of the most important questions in economics. At the same time, they are highly interesting for mathematicians because their structure is often quite difficult. We present a number of examples of such PDEs, discuss what is known about their properties, and list some open questions for future research.

scholarly journals Construction of elliptic stochastic partial differential equations solver in groundwater flow with convolutional neural networks

2021 ◽  
Vol 2083 (4) ◽  
pp. 042064
Author(s):  
Xue Pang ◽  
Jian Wang ◽  
Faliang Yin ◽  
Jun Yao
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Partial Differential Equations ◽  
Differential Equations ◽  
Boundary Conditions ◽  
Groundwater Flow ◽  
Convolutional Neural Networks ◽  
Stochastic Partial Differential Equations ◽  
Gaussian Random Field ◽  
Partial Differential

Abstract Elliptic stochastic partial differential equations (SPDEs) play an indispensable role in mathematics, engineering and other fields, and its solution methods emerge in endlessly with the progress of science and technology. In this paper, we make use of the convolutional neural networks (CNNs), which are widely used in machine learning, to construct a solver for SPDEs. The SPDEs with Neumann boundary conditions are considered, and two CNNs are employed. One is used to deal with the essential equation, and the other satisfies the boundary conditions. With the help of the length factor, the integrated neural network model can predict the solution of the equations accurately. We show an example of groundwater flow to evaluate the model proposed with Gaussian random field (GRF). The experimental results show that the proposed neural network solver can approximate the traditional numerical algorithm, and has high computational efficiency.

scholarly journals Discretization and machine learning approximation of BSDEs with a constraint on the Gains-process

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Idris Kharroubi ◽  
Thomas Lim ◽  
Xavier Warin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Differential Equations ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Learning Approach ◽  
The Neural Network ◽  
Machine Learning Approach ◽  
Mesh Grid

AbstractWe study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to zero. We then focus on the approximation of the discretely constrained BSDE. For that we adopt a machine learning approach. We show that the facelift can be approximated by an optimization problem over a class of neural networks under constraints on the neural network and its derivative. We then derive an algorithm converging to the discretely constrained BSDE as the number of neurons goes to infinity. We end by numerical experiments.

Sign in / Sign up

Export Citation Format

Share Document