scholarly journals <TT>sympl</TT> (v. 0.3.2) and <TT>climt</TT> (v. 0.11.0) – Towards a flexible framework for building model hierarchies in Python

2018 ◽  
Author(s):  
Joy Merwin Monteiro ◽  
Jeremy McGibbon ◽  
Rodrigo Caballero
Keyword(s):  
Data Analysis ◽  
Model Building ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Online Data ◽  
Wide Audience ◽  
Building Models ◽  
Model Components ◽  
The Individual

Abstract. sympl (System for Modelling Planets) and climt (Climate Modelling and diagnostics Toolkit) represent an attempt to rethink climate modelling frameworks from the ground up. The aim is to use expressive data structures available in the scientific Python ecosystem along with best practices in software design to build models that are self-documenting, highly inter-operable and that provide fine grained control over model components and behaviour. We believe that such an approach towards building models is essential to allow scientists to easily and reliably combine model components to represent the climate system at a desired level of complexity, and to enable users to fully understand what the model is doing. sympl is a framework which formulates the model in terms of a "state" which gets evolved forward in time by TimeStepper and Implicit components, and which can be modified by Diagnostic components. TimeStepper components in turn rely on Prognostic components to compute tendencies. Components contain all the information about the kinds of inputs they expect and outputs that they provide. Components can be used interchangeably, even when they rely on different units or array configurations. sympl provides basic functions and objects which could be used by any type of Earth system model. climt is an Earth system modelling toolkit that contains scientific components built over the sympl base objects. Components can be written in any language accessible from Python, and Fortran/C libraries are accessed via Cython. climt aims to provide different user APIs which trade-off simplicity of use against flexibility of model building, thus appealing to a wide audience. Model building, configuration and execution is through a Python script (or Jupyter Notebook), enabling researchers to build an end-to-end Python based pipeline along with popular Python based data analysis tools. Because of the modularity of the individual components, using online data analysis, visualisation or assimilation algorithms and tools with sympl/climt components is extremely simple.

scholarly journals sympl (v. 0.4.0) and climt (v. 0.15.3) – towards a flexible framework for building model hierarchies in Python

2018 ◽  
Vol 11 (9) ◽  
pp. 3781-3794 ◽  
Author(s):  
Joy Merwin Monteiro ◽  
Jeremy McGibbon ◽  
Rodrigo Caballero
Keyword(s):  
Model Building ◽  
Relevant Information ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Wide Audience ◽  
Fine Grained ◽  
Building Model ◽  
Building Models ◽  
Model Components

Abstract. sympl (System for Modelling Planets) and climt (Climate Modelling and Diagnostics Toolkit) are an attempt to rethink climate modelling frameworks from the ground up. The aim is to use expressive data structures available in the scientific Python ecosystem along with best practices in software design to allow scientists to easily and reliably combine model components to represent the climate system at a desired level of complexity and to enable users to fully understand what the model is doing. sympl is a framework which formulates the model in terms of a state that gets evolved forward in time or modified within a specific time by well-defined components. sympl's design facilitates building models that are self-documenting, are highly interoperable, and provide fine-grained control over model components and behaviour. sympl components contain all relevant information about the input they expect and output that they provide. Components are designed to be easily interchanged, even when they rely on different units or array configurations. sympl provides basic functions and objects which could be used in any type of Earth system model. climt is an Earth system modelling toolkit that contains scientific components built using sympl base objects. These include both pure Python components and wrapped Fortran libraries. climt provides functionality requiring model-specific assumptions, such as state initialization and grid configuration. climt's programming interface designed to be easy to use and thus appealing to a wide audience. Model building, configuration and execution are performed through a Python script (or Jupyter Notebook), enabling researchers to build an end-to-end Python-based pipeline along with popular Python data analysis and visualization tools.

ESM-Tools: A common infrastructure for modular coupled earth system modelling

2020 ◽  
Author(s):  
Dirk Barbi ◽  
Nadine Wieters ◽  
Luisa Cristini ◽  
Paul Gierz ◽  
Sara Khosravi ◽  
...  
Keyword(s):  
Ice Sheet ◽  
Modular System ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Isostatic Adjustment ◽  
Full Control ◽  
Model Components ◽  
User Friendly ◽  
Component Models

&lt;p&gt;Earth system and climate modelling involves the simulation of processes on a large range of scales, and within very different components of the earth system. In practice, component models from different institutes are mostly developed independently, and then combined using a dedicated coupling software.&lt;/p&gt;&lt;p&gt;This procedure not only leads to a wildly growing number of available versions of model components as well as coupled setups, but also to a specific way of obtaining and operating many of these. This can be a challenging problem (and potentially a huge waste of time) especially for unexperienced researchers, or scientists aiming to change to a different model system, e.g. for intercomparisons.&lt;/p&gt;&lt;p&gt;In order to define a standard way of downloading, configuring, compiling and running modular ESMs on a variety of HPC systems, AWI and partner institutions develop and maintain the OpenSource ESM-Tools software (https://www.esm-tools.net). Our aim is to provide standard solutions to typical problems occurring within the workflow of model simulations such as calendar operations, data postprocessing and monitoring, sanity checks, sorting and archiving of output, and script-based coupling (e.g. ice sheet models, isostatic adjustment models). The user only provides a short (30-40 lines) runscript of absolutely necessary experiment specific definitions, while the ESM-Tools execute the phases of a simulation in the correct order. A user-friendly API ensures that more experienced users have full control over each of these phases, and can easily add functionality. A GUI has been developed to provide a more intuitive approach to the modular system, and also to add a graphical overview over the available models and combinations.&lt;/p&gt;&lt;p&gt;Since revision 2 (released on March 19&lt;sup&gt;th&lt;/sup&gt; 2019), the ESM-Tools were entirely re-written, separating the implementation of actions (written in Python 3) from any information that we have, either on models, coupled setups, software tools, HPC systems etc. into nicely structured yaml configuration files. This has been done to reduce maintenance problems, and also to ensure that also unexperienced scientist can easily edit configurations, or even add new models or software without any programming experience. Since revision 3 the ESM-Tools support four ocean models (FESOM1, FESOM2, NEMO, MPIOM), three atmosphere models (ECHAM6, OpenIFS, ICON), two BGC models (HAMOCC, REcoM), an ice sheet (PISM) and an isostatic adjustment model (VILMA) as well as standard settings for five HPC systems. For the future we plan to add interfaces to regional models and soil/hydrology models.&lt;/p&gt;&lt;p&gt;The Tools currently have more than 70 registered users from 5 institutions, and more than 40 authors of contributions to either model configurations or functionality.&lt;/p&gt;

scholarly journals Towards in-situ visualization integrated earth system models: RegESM 1.1 regional modelling system

2018 ◽  
Author(s):  
Ufuk Utku Turuncoglu
Keyword(s):  
Data Analysis ◽  
System Model ◽  
Earth System ◽  
System Models ◽  
Modeling Systems ◽  
Temporal And Spatial ◽  
Model Components ◽  
Earth System Models ◽  
Modelling System

Abstract. The data volume being produced by regional and global multi-component earth system models are rapidly increasing due to the improved spatial and temporal resolution of the model components, sophistication of the used numerical models in terms of represented physical processes and their non-linear complex interactions. In particular, very short time steps have to be defined in multi-component and multi-scale non-hydrostatic modelling systems to represent the evolution of the fast-moving processes such as turbulence, extra-tropical cyclones, convective lines, jet streams, internal waves, vertical turbulent mixing and surface gravity waves. Consequently, the used small time steps cause extra computation and disk I/O overhead in the used modelling system even if today's most powerful high-performance computing and data storage systems are being considered. Analysis of the high volume of data from multiple earth system model components at different temporal and spatial resolution also poses a challenging problem to efficiently perform integrated data analysis of the massive amounts of data by relying on the conventional post-processing methods available today. This study basically aims to explore the feasibility and added value of integrating existing in-situ visualization and data analysis methods with the model coupling framework (ESMF) to increase interoperability between multi-component simulation code and data processing pipelines by providing easy to use, efficient, generic and standardized modeling environment for earth system science applications. The new data analysis approach enables simultaneous analysis of the vast amount of data produced by multi-component regional earth system models (atmosphere, ocean etc.) during the run process. The methodology aims to create an integrated modeling environment for analyzing fast-moving processes and their evolution in both time and space to support better understanding of the underplaying physical mechanisms. The state-of-art approach can also be used to solve common problems in earth system model development workflow such as designing new sub-grid scale parametrizations (convection, air–sea interaction etc.) that requires inspecting the integrated model behavior in a higher temporal and spatial scale during the run or supporting visual debugging of the multi-component modeling systems, which usually are not facilitated by existing model coupling libraries and modeling systems.

scholarly journals Design and implementation of the infrastructure of HadGEM3: the next-generation Met Office climate modelling system

2011 ◽  
Vol 4 (2) ◽  
pp. 223-253 ◽  
Author(s):  
H. T. Hewitt ◽  
D. Copsey ◽  
I. D. Culverwell ◽  
C. M. Harris ◽  
R. S. R. Hill ◽  
...  
Keyword(s):  
Climate Model ◽  
Model Performance ◽  
Model Atmosphere ◽  
Ocean Model ◽  
Climate Modelling ◽  
Model Components ◽  
The Individual ◽  
The Uk ◽  
Technical Solutions ◽  
Modelling System

Abstract. This paper describes the development of a technically robust climate modelling system, HadGEM3, which couples the Met Office Unified Model atmosphere component, the NEMO ocean model and the Los Alamos sea ice model (CICE) using the OASIS coupler. Details of the coupling and technical solutions of the physical model (HadGEM3-AO) are documented, in addition to a description of the configurations of the individual submodels. The paper demonstrates that the implementation of the model has resulted in accurate conservation of heat and freshwater across the model components. The model performance in early versions of this climate model is briefly described to demonstrate that the results are scientifically credible. HadGEM3-AO is the basis for a number of modelling efforts outside of the Met Office, both within the UK and internationally. This documentation of the HadGEM3-AO system provides a detailed reference for developers of HadGEM3-based climate configurations.

Modular AWI-CM: An Earth System Model (ESM) prototype using the esm-interface library for a modular ESM coupling approach

2020 ◽  
Author(s):  
Nadine Wieters ◽  
Dirk Barbi ◽  
Luisa Cristini
Keyword(s):  
Climate Model ◽  
General Idea ◽  
Second Step ◽  
System Modelling ◽  
Earth System ◽  
First Results ◽  
Coupling Approach ◽  
Domain Models ◽  
Model Components ◽  
Earth System Models

&lt;p&gt;Earth System Models (ESMs) are composed of different components, including submodels as well as whole domain models. Within such an ESM, these model components need to exchange information to account for the interactions between the different compartments. This exchange of data is the purpose of a &amp;#8220;model coupler&amp;#8221;.&lt;/p&gt;&lt;p&gt;Within the Advanced Earth System Modelling Capacity (ESM) project, a goal is to develop a modular framework that allows for a flexible ESM configuration. One approach is to implement purpose build model couplers in a more modular way.&lt;/p&gt;&lt;p&gt;For this purpose, we developed the esm-interface library, in consideration of the following objectives: (i) To obtain a more modular ESM, that allows model components and model couplers to be exchangeable; and (ii) to account for a more flexible coupling configuration of an ESM setup.&lt;/p&gt;&lt;p&gt;As a first application of the esm-interface library, we implemented it into the AWI Climate Model (AWI-CM) [Sidorenko et al., 2015] as an interface between the model components and the model coupler (OASIS3-MCT; Valcke [2013]). In a second step, we extended the esm-interface library for a second coupler (YAC; Hanke et al. [2016]).&lt;/p&gt;&lt;p&gt;In this presentation, we will discuss the general idea of the esm-interface library, it&amp;#8217;s implementation in an ESM setup and show first results from the first modular prototype of AWI-CM.&lt;/p&gt;

scholarly journals Coupling technologies for Earth System Modelling

2012 ◽  
Vol 5 (3) ◽  
pp. 1987-2006 ◽  
Author(s):  
S. Valcke ◽  
V. Balaji ◽  
A. Craig ◽  
C. DeLuca ◽  
R. Dunlap ◽  
...  
Keyword(s):  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Common Features ◽  
The Earth ◽  
Future Challenges ◽  
Computing Platforms ◽  
Design Characteristics ◽  
Earth System Modelling

Abstract. This paper presents a review of the software currently used in climate modelling in general and in CMIP5 in particular to couple the numerical codes representing the different components of the Earth system. The coupling technologies presented show common features, such as the ability to communicate and regrid data, but also offer different functions and implementations. Design characteristics of the different approaches are discussed as well as future challenges arising from the increasing complexity of scientific problems and computing platforms.

scholarly journals ESM-Tools Version 4.0: A modular infrastructure for stand-alone and coupled Earth System Modelling (ESM)

2020 ◽  
Author(s):  
Dirk Barbi ◽  
Nadine Wieters ◽  
Paul Gierz ◽  
Fatemeh Chegini ◽  
Sara Khosravi ◽  
...  
Keyword(s):  
Land Surface ◽  
High Performance ◽  
Surface Model ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Coupled Models ◽  
Wide Range ◽  
Runtime Infrastructure ◽  
Earth System Modelling

Abstract. Earth system and climate modelling involves the simulation of processes on a wide range of scales and within and across various components of the Earth system. In practice, component models are often developed independently by different research groups and then combined using a dedicated coupling software. This procedure not only leads to a strongly growing number of available versions of model components and coupled setups but also to model- and system-dependent ways of obtaining and operating them. Therefore, implementing these Earth System Models (ESMs) can be challenging and extremely time-consuming, especially for less experienced modellers, or scientists aiming to use different ESMs as in the case of inter-comparison projects. To assist researchers and modellers by reducing avoidable complexity, we developed the ESM-Tools software, which provides a standard way for downloading, configuring, compiling, running and monitoring different models - coupled ESMs and stand-alone models alike - on a variety of High-Performance Computing (HPC) systems. (The ESM-Tools are equally applicable and helpful for stand-alone as for coupled models. In fact, the ESM-Tools are used as standard compile and runtime infrastructure for FESOM2, and currently also applied for ECHAM and ICON standalone simulations. As coupled ESMs are technically the more challenging tasks, we will focus on coupled setups, always implying that stand-alone models can benefit in the same way.) With the ESM-Tools, the user is only required to provide a short script consisting of only the experiment specific definitions, while the software executes all the phases of a simulation in the correct order. The software, which is well documented and easy to install and use, currently supports four ocean models, three atmosphere models, two biogeochemistry models, an ice sheet model, an isostatic adjustment model, a hydrology model and a land-surface model. ESM-Tools has been entirely re-coded in a high-level programming language (Python) and provides researchers with an even more user-friendly interface for Earth system modelling lately. The ESM-Tools were developed within the framework of the project Advanced Earth System Model Capacity, supported by the Helmholtz Association.

Earth System Modelling and Data Analysis

2012 ◽  
pp. 57-82
Author(s):  
Gerrit Lohmann ◽  
Klaus Grosfeld ◽  
Dieter Wolf-Gladrow ◽  
Anna Wegner ◽  
Justus Notholt ◽  
...  
Keyword(s):  
Data Analysis ◽  
System Modelling ◽  
Earth System ◽  
Earth System Modelling

scholarly journals YAC 1.2.0: An extendable coupling software for Earth system modelling

2016 ◽  
Author(s):  
M. Hanke ◽  
R. Redler ◽  
T. Holfeld ◽  
M. Yastremsky
Keyword(s):  
General Circulation ◽  
A Priori ◽  
Circulation Model ◽  
System Model ◽  
System Modelling ◽  
Earth System ◽  
Parallel Performance ◽  
Neighbourhood Search ◽  
Physical Fields ◽  
Model Components

Abstract. A light-weight software framework has been developed as a library to realise the coupling of Earth system model components. The software provides a parallelised 2-dimensional neighbourhood search, interpolation, and communication for the coupling between any two model components. The software offers flexible coupling of physical fields defined on regular and irregular grids on the sphere without a priori assumptions about the particular grid structure or grid element types. All supported grids can be combined with any of the supported interpolations. We describe our approach and provide an overview about some of the algorithms we are using and the implemented functionality. The parallel performance is examined with a set of realistic use cases. The coupling software is now used for the coupling of the model components in the Icosahedral nonhydrostatic (ICON) general circulation model.

scholarly journals Coupling technologies for Earth System Modelling

2012 ◽  
Vol 5 (6) ◽  
pp. 1589-1596 ◽  
Author(s):  
S. Valcke ◽  
V. Balaji ◽  
A. Craig ◽  
C. DeLuca ◽  
R. Dunlap ◽  
...  
Keyword(s):  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Common Features ◽  
The Earth ◽  
Future Challenges ◽  
Computing Platforms ◽  
Design Characteristics ◽  
Earth System Modelling

Abstract. This paper presents a review of the software currently used in climate modelling in general and in CMIP5 in particular to couple the numerical codes representing the different components of the Earth System. The coupling technologies presented show common features, such as the ability to communicate and regrid data, and also offer different functions and implementations. Design characteristics of the different approaches are discussed as well as future challenges arising from the increasing complexity of scientific problems and computing platforms.

Sign in / Sign up

Export Citation Format

Share Document