scholarly journals Mini-batch optimization enables training of ODE models on large-scale datasets

2022 ◽  
Vol 13 (1) ◽  
Author(s):  
Paul Stapor ◽  
Leonard Schmiester ◽  
Christoph Wierling ◽  
Simon Merkt ◽  
Dilan Pathirana ◽  
...  
Keyword(s):  
Large Scale ◽  
Dynamic Models ◽  
Dynamic Modelling ◽  
Optimization Methods ◽  
Scale Model ◽  
Model Parameters ◽  
Scaling Properties ◽  
Ode Models ◽  
Cellular Signal ◽  
Order Of Magnitude

AbstractQuantitative dynamic models are widely used to study cellular signal processing. A critical step in modelling is the estimation of unknown model parameters from experimental data. As model sizes and datasets are steadily growing, established parameter optimization approaches for mechanistic models become computationally extremely challenging. Mini-batch optimization methods, as employed in deep learning, have better scaling properties. In this work, we adapt, apply, and benchmark mini-batch optimization for ordinary differential equation (ODE) models, thereby establishing a direct link between dynamic modelling and machine learning. On our main application example, a large-scale model of cancer signaling, we benchmark mini-batch optimization against established methods, achieving better optimization results and reducing computation by more than an order of magnitude. We expect that our work will serve as a first step towards mini-batch optimization tailored to ODE models and enable modelling of even larger and more complex systems than what is currently possible.

scholarly journals Mini-batch optimization enables training of ODE models on large-scale datasets

2019 ◽  
Author(s):  
Paul Stapor ◽  
Leonard Schmiester ◽  
Christoph Wierling ◽  
Bodo M.H. Lange ◽  
Daniel Weindl ◽  
...  
Keyword(s):  
Large Scale ◽  
Optimization Methods ◽  
Scale Model ◽  
Model Parameters ◽  
Dynamical Models ◽  
Scaling Properties ◽  
Ode Models ◽  
Large Scale Model ◽  
Cellular Signal ◽  
Order Of Magnitude

AbstractQuantitative dynamical models are widely used to study cellular signal processing. A critical step in modeling is the estimation of unknown model parameters from experimental data. As model sizes and datasets are steadily growing, established parameter optimization approaches for mechanistic models become computationally extremely challenging. However, mini-batch optimization methods, as employed in deep learning, have better scaling properties. In this work, we adapt, apply, and benchmark mini-batch optimization for ordinary differential equation (ODE) models thereby establishing a direct link between dynamic modeling and machine learning. On our main application example, a large-scale model of cancer signaling, we benchmark mini-batch optimization against established methods, achieving better optimization results and reducing computation by more than an order of magnitude. We expect that our work will serve as a first step towards mini-batch optimization tailored to ODE models and enable modeling of even larger and more complex systems than what is currently possible.

scholarly journals Constraints on aerosol processes in climate models from vertically-resolved aircraft observations of black carbon

2013 ◽  
Vol 13 (1) ◽  
pp. 437-473 ◽  
Author(s):  
Z. Kipling ◽  
P. Stier ◽  
J. P. Schwarz ◽  
A. E. Perring ◽  
J. R. Spackman ◽  
...  
Keyword(s):  
Vertical Distribution ◽  
Black Carbon ◽  
Biomass Burning ◽  
Large Scale ◽  
Climate Models ◽  
Convective Transport ◽  
Convective Precipitation ◽  
Scale Model ◽  
Modest Improvement ◽  
Order Of Magnitude

Abstract. Evaluation of the aerosol schemes in current climate models is dependent upon the available observational data. In-situ observations from flight campaigns can provide valuable data about the vertical distribution of aerosol that is difficult to obtain from satellite or ground-based platforms, although they are localised in space and time. Using single-particle soot-photometer (SP2) measurements from the HIAPER Pole-to-Pole Observations (HIPPO) campaign, which consists of many vertical profiles over a large region of the Pacific, we evaluate the meridional and vertical distribution of black carbon (BC) aerosol simulated by the HadGEM3-UKCA and ECHAM5-HAM2 models. Both models show a similar pattern of overestimating the BC column burden compared to that derived from the observations, in many areas by an order of magnitude. However, by sampling the simulated BC mass mixing ratio along the flight track and comparing to the observations, we show that this discrepancy has a rather different vertical structure in the two models. Using this methodology, we conduct sensitivity tests on two specific elements of the models: biomass-burning emissions and scavenging by convective precipitation. We show that, by coupling the convective scavenging more tightly with convective transport, both the column burden and vertical distribution of BC in HadGEM3–UKCA are significantly improved with respect to the observations, demonstrating the importance of a realistic representation of this process. In contrast, updating from GFED2 to GFED3.1 biomass-burning emissions makes a more modest improvement in both models, which is not statistically significant. We also demonstrate the important role that nudged simulations (where the large-scale model dynamics are continuously relaxed towards a reanalysis) can play in this type of evaluation, allowing statistically significant differences between configurations of the aerosol scheme to be seen where the differences between the corresponding free-running simulations would not be significant.

scholarly journals Quantifying the effect of ocean bed properties on ice sheet geometry over 40 000 years with a full-Stokes model

2020 ◽  
Vol 14 (11) ◽  
pp. 3917-3934
Author(s):  
Clemens Schannwell ◽  
Reinhard Drews ◽  
Todd A. Ehlers ◽  
Olaf Eisen ◽  
Christoph Mayer ◽  
...  
Keyword(s):  
Dynamic Models ◽  
Ice Sheet ◽  
Time Frame ◽  
Model Parameters ◽  
Glacial Cycle ◽  
Ice Shelf ◽  
Glacial Cycles ◽  
Sediment Redistribution ◽  
Grounding Line ◽  
Order Of Magnitude

Abstract. Simulations of ice sheet evolution over glacial cycles require integration of observational constraints using ensemble studies with fast ice sheet models. These include physical parameterisations with uncertainties, for example, relating to grounding-line migration. More complete ice dynamic models are slow and have thus far only be applied for < 1000 years, leaving many model parameters unconstrained. Here we apply a 3D thermomechanically coupled full-Stokes ice sheet model to the Ekström Ice Shelf embayment, East Antarctica, over a full glacial cycle (40 000 years). We test the model response to differing ocean bed properties that provide an envelope of potential ocean substrates seawards of today's grounding line. The end-member scenarios include a hard, high-friction ocean bed and a soft, low-friction ocean bed. We find that predicted ice volumes differ by > 50 % under almost equal forcing. Grounding-line positions differ by up to 49 km, show significant hysteresis, and migrate non-steadily in both scenarios with long quiescent phases disrupted by leaps of rapid migration. The simulations quantify the evolution of two different ice sheet geometries (namely thick and slow vs. thin and fast), triggered by the variable grounding-line migration over the differing ocean beds. Our study extends the timescales of 3D full-Stokes by an order of magnitude compared to previous studies with the help of parallelisation. The extended time frame for full-Stokes models is a first step towards better understanding other processes such as erosion and sediment redistribution in the ice shelf cavity impacting the entire catchment geometry.

scholarly journals Patient-specific logic models of signaling pathways from screenings on cancer biopsies to prioritize personalized combination therapies

2018 ◽  
Author(s):  
Federica Eduati ◽  
Patricia Jaaks ◽  
Christoph A. Merten ◽  
Mathew J. Garnett ◽  
Julio Saez- Rodriguez
Keyword(s):  
Signaling Pathways ◽  
Large Scale ◽  
Dynamic Models ◽  
Ex Vivo ◽  
Response To Therapy ◽  
Mechanistic Modeling ◽  
Patient Specific ◽  
Model Parameters ◽  
Specific Response ◽  
Tumor Stages

AbstractMechanistic modeling of signaling pathways mediating patient-specific response to therapy can help to unveil resistance mechanisms and improve therapeutic strategies. Yet, creating such models for patients, in particular for solid malignancies, is challenging. A major hurdle to build these models is the limited material available, that precludes the generation of large-scale perturbation data. Here, we present an approach that couples ex vivo high-throughput screenings of cancer biopsies using microfluidics with logic-based modeling to generate patient-specific dynamic models of extrinsic and intrinsic apoptosis signaling pathways. We used the resulting models to investigate heterogeneity in pancreatic cancer patients, showing dissimilarities especially in the PI3K-Akt pathway. Variation in model parameters reflected well the different tumor stages. Finally, we used our dynamic models to efficaciously predict new personalized combinatorial treatments. Our results suggest our combination of microfluidic experiments and mathematical model can be a novel tool toward cancer precision medicine.

scholarly journals Online dynamical downscaling of temperature and precipitation within the <i>i</i>LOVECLIM model (version 1.1)

2018 ◽  
Vol 11 (1) ◽  
pp. 453-466
Author(s):  
Aurélien Quiquet ◽  
Didier M. Roche ◽  
Christophe Dumas ◽  
Didier Paillard
Keyword(s):  
High Resolution ◽  
Land Surface ◽  
Large Scale ◽  
Climate Model ◽  
Dynamical Downscaling ◽  
Atmospheric Model ◽  
Scale Model ◽  
Model Parameters ◽  
Temperature And Precipitation ◽  
Large Scale Model

Abstract. This paper presents the inclusion of an online dynamical downscaling of temperature and precipitation within the model of intermediate complexity iLOVECLIM v1.1. We describe the following methodology to generate temperature and precipitation fields on a 40 km  ×  40 km Cartesian grid of the Northern Hemisphere from the T21 native atmospheric model grid. Our scheme is not grid specific and conserves energy and moisture in the same way as the original climate model. We show that we are able to generate a high-resolution field which presents a spatial variability in better agreement with the observations compared to the standard model. Although the large-scale model biases are not corrected, for selected model parameters, the downscaling can induce a better overall performance compared to the standard version on both the high-resolution grid and on the native grid. Foreseen applications of this new model feature include the improvement of ice sheet model coupling and high-resolution land surface models.

Estimation of distributed parameters at regional scale by history-matching of a multi-layered sedimentary aquifer

2020 ◽  
Author(s):  
Ryma Aissat ◽  
Alexandre Pryet ◽  
Marc Saltel ◽  
Alain Dupuy
Keyword(s):  
Parameter Estimation ◽  
Hydraulic Properties ◽  
Large Scale ◽  
Classical Method ◽  
Regional Scale ◽  
Scale Model ◽  
Estimation Methods ◽  
Model Parameters ◽  
Convergence Criteria ◽  
Distributed Parameters

&lt;p&gt;Large scale, physically-based groundwater models have been used for many years for water resources management and decision-support. Improving the accuracy and reliability of these models is a constant objective. The characterization of model parameters, in particular hydraulic properties, which are spatially heterogeneous is a challenge. Parameter estimation algorithms can now manage numerous model runs in parallel, but the operation remains, in practice, largely constrained by the computational burden. A large-scale model of the sedimentary, multilayered aquifer system of North Aquitania (MONA), in South-West France, developed by the French Geological Survey (BRGM) is used here to illustrate the case. We focus on the estimation of distributed parameters and investigate the optimum parameterization given the level of spatial heterogeneity we aim to characterize, available observations, model run time, and computational resources. Hydraulic properties are estimated with pilot points. Interpolation is conducted by kriging, the variogram range and pilot point density are set given modeling purposes and a series of constraints. The popular gradient-based parameter estimation methods such as the Gauss&amp;#8211;Marquard&amp;#8211;Levenberg algorithm (GLMA) are conditioned by the integrity of the Jacobian matrix. We investigate the trade-off between strict convergence criteria, which insure a better integrity of derivatives, and loose convergence criteria, which reduce computation time. The results obtained with the classical method (GLMA) are compared with the results of an emerging method, the Iterative Ensemble Smoother (IES). Some guidelines are eventually provided for parameter estimation of large-scale multi-layered groundwater models.&lt;/p&gt;

scholarly journals Efficient parameterization of large-scale dynamic models based on relative measurements

2019 ◽  
Author(s):  
Leonard Schmiester ◽  
Yannik Schälte ◽  
Fabian Fröhlich ◽  
Jan Hasenauer ◽  
Daniel Weindl
Keyword(s):  
Large Scale ◽  
Dynamic Models ◽  
Error Model ◽  
Supplementary Information ◽  
Reaction Networks ◽  
Model Parameters ◽  
Biological Processes ◽  
Wide Range ◽  
Scale Models ◽  
Relative Measurements

Abstract Motivation Mechanistic models of biochemical reaction networks facilitate the quantitative understanding of biological processes and the integration of heterogeneous datasets. However, some biological processes require the consideration of comprehensive reaction networks and therefore large-scale models. Parameter estimation for such models poses great challenges, in particular when the data are on a relative scale. Results Here, we propose a novel hierarchical approach combining (i) the efficient analytic evaluation of optimal scaling, offset and error model parameters with (ii) the scalable evaluation of objective function gradients using adjoint sensitivity analysis. We evaluate the properties of the methods by parameterizing a pan-cancer ordinary differential equation model (>1000 state variables, >4000 parameters) using relative protein, phosphoprotein and viability measurements. The hierarchical formulation improves optimizer performance considerably. Furthermore, we show that this approach allows estimating error model parameters with negligible computational overhead when no experimental estimates are available, providing an unbiased way to weight heterogeneous data. Overall, our hierarchical formulation is applicable to a wide range of models, and allows for the efficient parameterization of large-scale models based on heterogeneous relative measurements. Availability and implementation Supplementary code and data are available online at http://doi.org/10.5281/zenodo.3254429 and http://doi.org/10.5281/zenodo.3254441. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Modelling and Forecasting Domestic Tourism. Case Study from Armenia

2021 ◽  
Vol 5 (2) ◽  
pp. 96-110
Author(s):  
Karen Poghosyan ◽  
Gayane Tovmasyan
Keyword(s):  
Large Scale ◽  
Dynamic Models ◽  
Dynamic Modelling ◽  
Tourism Development ◽  
Point Of View ◽  
Sufficient Evidence ◽  
Small Scale ◽  
Scale Models ◽  
Domestic Tourism ◽  
Modelling And Forecasting

This paper summarizes the arguments and counterarguments within the scientific discussion on the issue of modelling and forecasting domestic tourism. During Covid-19 many countries tried to develop domestic tourism as an alternative to inbound tourism. In Armenia domestic tourism has grown recently, and in 2020 the decrease was 33% compared to last year. The main purpose of the research is to model and forecast domestic tourism growth in Armenia. Systematization of the literary sources and approaches for solving the problem indicates that many models and different variables are used to forecast tourism development. Methodological tools of the research methods were static and dynamic models, years of research were 2001-2020, quarterly data. The paper presents the results of an empirical analysis, which showed that with the static regression analysis a 1% change in GDP will lead to a change of 4.43% in the number of domestic tourists, a 1% change in the CPI will lead to a 14.55% change in the number of domestic tourists. For dynamic modelling we used 12 competing short-term forecasting models. Based on the recursive and rolling forecast simulation results we concluded that out-of-sample forecasts obtained by the small-scale models outperform forecasts obtained by the large-scale models at all forecast horizons. So, the forecasts of the domestic tourists’ growth obtained by small-scale models are more appropriate from the practical point of view. Then, in order to check whether the differences in forecasts obtained by the different models are statistically significant we applied Diebold-Mariano test. Based on the results of this test we concluded that there is not sufficient evidence to favor large-scale over small-scale models. This means that the forecast results obtained for domestic tourist growth by using the small scale models would not be statistically different from the results obtained by the large scale models. Based on the analysis, the forecasted values for domestic tourists for the future years were determined. The results of the research can be useful for state bodies, as well as private organizations, and for everybody who wants to model and forecast tourism development.

scholarly journals Computational tools for inversion and uncertainty estimation in respirometry

PLoS ONE ◽  
2021 ◽  
Vol 16 (5) ◽  
pp. e0251926
Author(s):  
Taewon Cho ◽  
Hodjat Pendar ◽  
Julianne Chung
Keyword(s):  
Inverse Problem ◽  
Impulse Response ◽  
Response Function ◽  
Large Scale ◽  
Sampling Rate ◽  
Impulse Response Function ◽  
Optimization Methods ◽  
Model Parameters ◽  
Computational Tools ◽  
Living Organisms

In many physiological systems, real-time endogeneous and exogenous signals in living organisms provide critical information and interpretations of physiological functions; however, these signals or variables of interest are not directly accessible and must be estimated from noisy, measured signals. In this paper, we study an inverse problem of recovering gas exchange signals of animals placed in a flow-through respirometry chamber from measured gas concentrations. For large-scale experiments (e.g., long scans with high sampling rate) that have many uncertainties (e.g., noise in the observations or an unknown impulse response function), this is a computationally challenging inverse problem. We first describe various computational tools that can be used for respirometry reconstruction and uncertainty quantification when the impulse response function is known. Then, we address the more challenging problem where the impulse response function is not known or only partially known. We describe nonlinear optimization methods for reconstruction, where both the unknown model parameters and the unknown signal are reconstructed simultaneously. Numerical experiments show the benefits and potential impacts of these methods in respirometry.

scholarly journals Online dynamical downscaling of temperature and precipitation within the iLOVECLIM model (version 1.1)

2017 ◽  
Author(s):  
Aurélien Quiquet ◽  
Didier M. Roche ◽  
Christophe Dumas ◽  
Didier Paillard
Keyword(s):  
High Resolution ◽  
Land Surface ◽  
Large Scale ◽  
Dynamical Downscaling ◽  
Land Surface Model ◽  
Atmospheric Model ◽  
Scale Model ◽  
Surface Model ◽  
Model Parameters ◽  
Temperature And Precipitation

Abstract. In this paper, we present the inclusion of an online dynamical downscaling of heat and moisture within the model of intermediate complexity iLOVECLIM v1.1. We describe the followed methodology to generate temperature and precipitation fields on a 40 km × 40 km Cartesian grid of the Northern Hemisphere from the T21 native atmospheric model grid. Our scheme is non grid-specific and conserves energy and moisture. We show that we are able to generate a high resolution field which presents a spatial variability in better agreement with the observations compared to the standard model. Whilst the large-scale model biases are not corrected, for selected model parameters, the downscaling can induce a better overall performance compared to the standard version on both the high-resolution grid and on the native grid. Foreseen applications of this new model feature includes ice sheet model coupling and high-resolution land surface model.

Sign in / Sign up

Export Citation Format

Share Document