ESM-Tools: A common infrastructure for modular coupled earth system modelling

2020 ◽  
Author(s):  
Dirk Barbi ◽  
Nadine Wieters ◽  
Luisa Cristini ◽  
Paul Gierz ◽  
Sara Khosravi ◽  
...  
Keyword(s):  
Ice Sheet ◽  
Modular System ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Isostatic Adjustment ◽  
Full Control ◽  
Model Components ◽  
User Friendly ◽  
Component Models

<p>Earth system and climate modelling involves the simulation of processes on a large range of scales, and within very different components of the earth system. In practice, component models from different institutes are mostly developed independently, and then combined using a dedicated coupling software.</p><p>This procedure not only leads to a wildly growing number of available versions of model components as well as coupled setups, but also to a specific way of obtaining and operating many of these. This can be a challenging problem (and potentially a huge waste of time) especially for unexperienced researchers, or scientists aiming to change to a different model system, e.g. for intercomparisons.</p><p>In order to define a standard way of downloading, configuring, compiling and running modular ESMs on a variety of HPC systems, AWI and partner institutions develop and maintain the OpenSource ESM-Tools software (https://www.esm-tools.net). Our aim is to provide standard solutions to typical problems occurring within the workflow of model simulations such as calendar operations, data postprocessing and monitoring, sanity checks, sorting and archiving of output, and script-based coupling (e.g. ice sheet models, isostatic adjustment models). The user only provides a short (30-40 lines) runscript of absolutely necessary experiment specific definitions, while the ESM-Tools execute the phases of a simulation in the correct order. A user-friendly API ensures that more experienced users have full control over each of these phases, and can easily add functionality. A GUI has been developed to provide a more intuitive approach to the modular system, and also to add a graphical overview over the available models and combinations.</p><p>Since revision 2 (released on March 19<sup>th</sup> 2019), the ESM-Tools were entirely re-written, separating the implementation of actions (written in Python 3) from any information that we have, either on models, coupled setups, software tools, HPC systems etc. into nicely structured yaml configuration files. This has been done to reduce maintenance problems, and also to ensure that also unexperienced scientist can easily edit configurations, or even add new models or software without any programming experience. Since revision 3 the ESM-Tools support four ocean models (FESOM1, FESOM2, NEMO, MPIOM), three atmosphere models (ECHAM6, OpenIFS, ICON), two BGC models (HAMOCC, REcoM), an ice sheet (PISM) and an isostatic adjustment model (VILMA) as well as standard settings for five HPC systems. For the future we plan to add interfaces to regional models and soil/hydrology models.</p><p>The Tools currently have more than 70 registered users from 5 institutions, and more than 40 authors of contributions to either model configurations or functionality.</p>

scholarly journals sympl (v. 0.4.0) and climt (v. 0.15.3) – towards a flexible framework for building model hierarchies in Python

2018 ◽  
Vol 11 (9) ◽  
pp. 3781-3794 ◽  
Author(s):  
Joy Merwin Monteiro ◽  
Jeremy McGibbon ◽  
Rodrigo Caballero
Keyword(s):  
Model Building ◽  
Relevant Information ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Wide Audience ◽  
Fine Grained ◽  
Building Model ◽  
Building Models ◽  
Model Components

Abstract. sympl (System for Modelling Planets) and climt (Climate Modelling and Diagnostics Toolkit) are an attempt to rethink climate modelling frameworks from the ground up. The aim is to use expressive data structures available in the scientific Python ecosystem along with best practices in software design to allow scientists to easily and reliably combine model components to represent the climate system at a desired level of complexity and to enable users to fully understand what the model is doing. sympl is a framework which formulates the model in terms of a state that gets evolved forward in time or modified within a specific time by well-defined components. sympl's design facilitates building models that are self-documenting, are highly interoperable, and provide fine-grained control over model components and behaviour. sympl components contain all relevant information about the input they expect and output that they provide. Components are designed to be easily interchanged, even when they rely on different units or array configurations. sympl provides basic functions and objects which could be used in any type of Earth system model. climt is an Earth system modelling toolkit that contains scientific components built using sympl base objects. These include both pure Python components and wrapped Fortran libraries. climt provides functionality requiring model-specific assumptions, such as state initialization and grid configuration. climt's programming interface designed to be easy to use and thus appealing to a wide audience. Model building, configuration and execution are performed through a Python script (or Jupyter Notebook), enabling researchers to build an end-to-end Python-based pipeline along with popular Python data analysis and visualization tools.

scholarly journals <TT>sympl</TT> (v. 0.3.2) and <TT>climt</TT> (v. 0.11.0) – Towards a flexible framework for building model hierarchies in Python

2018 ◽  
Author(s):  
Joy Merwin Monteiro ◽  
Jeremy McGibbon ◽  
Rodrigo Caballero
Keyword(s):  
Data Analysis ◽  
Model Building ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Online Data ◽  
Wide Audience ◽  
Building Models ◽  
Model Components ◽  
The Individual

Abstract. sympl (System for Modelling Planets) and climt (Climate Modelling and diagnostics Toolkit) represent an attempt to rethink climate modelling frameworks from the ground up. The aim is to use expressive data structures available in the scientific Python ecosystem along with best practices in software design to build models that are self-documenting, highly inter-operable and that provide fine grained control over model components and behaviour. We believe that such an approach towards building models is essential to allow scientists to easily and reliably combine model components to represent the climate system at a desired level of complexity, and to enable users to fully understand what the model is doing. sympl is a framework which formulates the model in terms of a "state" which gets evolved forward in time by TimeStepper and Implicit components, and which can be modified by Diagnostic components. TimeStepper components in turn rely on Prognostic components to compute tendencies. Components contain all the information about the kinds of inputs they expect and outputs that they provide. Components can be used interchangeably, even when they rely on different units or array configurations. sympl provides basic functions and objects which could be used by any type of Earth system model. climt is an Earth system modelling toolkit that contains scientific components built over the sympl base objects. Components can be written in any language accessible from Python, and Fortran/C libraries are accessed via Cython. climt aims to provide different user APIs which trade-off simplicity of use against flexibility of model building, thus appealing to a wide audience. Model building, configuration and execution is through a Python script (or Jupyter Notebook), enabling researchers to build an end-to-end Python based pipeline along with popular Python based data analysis tools. Because of the modularity of the individual components, using online data analysis, visualisation or assimilation algorithms and tools with sympl/climt components is extremely simple.

scholarly journals OASIS4 – a coupling software for next generation earth system modelling

2009 ◽  
Vol 2 (2) ◽  
pp. 797-843 ◽  
Author(s):  
R. Redler ◽  
S. Valcke ◽  
H. Ritzdorf
Keyword(s):  
Individual Component ◽  
Search Algorithm ◽  
Coupled Model ◽  
System Modelling ◽  
Earth System ◽  
Core Technology ◽  
Primary Focus ◽  
Model Components ◽  
Component Models ◽  
Earth System Modelling

Abstract. In this article we present a new version of the Ocean Atmosphere Sea Ice Soil coupling software (OASIS4). With this new fully parallel OASIS4 coupler we target the needs of Earth system modelling in its full complexity. The primary focus of this article is to describe the design of the OASIS4 software and how the coupling software drives the whole coupled model system ensuring the synchronization of the different component models. The application programmer interface (API) manages the coupling exchanges between arbitrary climate component models, as well as the input and output from and to files of each individual component. The OASIS4 Transformer instance performs the parallel interpolation and transfer of the coupling data between source and target model components. As a new core technology for the software, the fully parallel search algorithm of OASIS4 is described in detail. First benchmark results are discussed with simple test configurations to demonstrate the efficiency and scalability of the software when applied to Earth system model components. Typically the compute time needed in order to perform the search is in the order of a few seconds and is only weakly dependant on the grid size.

scholarly journals OASIS4 – a coupling software for next generation earth system modelling

2010 ◽  
Vol 3 (1) ◽  
pp. 87-104 ◽  
Author(s):  
R. Redler ◽  
S. Valcke ◽  
H. Ritzdorf
Keyword(s):  
Individual Component ◽  
Search Algorithm ◽  
Coupled Model ◽  
System Modelling ◽  
Earth System ◽  
Core Technology ◽  
Primary Focus ◽  
Model Components ◽  
Component Models ◽  
Earth System Modelling

Abstract. In this article we present a new version of the Ocean Atmosphere Sea Ice Soil coupling software (OASIS4). With this new fully parallel OASIS4 coupler we target the needs of Earth system modelling in its full complexity. The primary focus of this article is to describe the design of the OASIS4 software and how the coupling software drives the whole coupled model system ensuring the synchronization of the different component models. The application programmer interface (API) manages the coupling exchanges between arbitrary climate component models, as well as the input and output from and to files of each individual component. The OASIS4 Transformer instance performs the parallel interpolation and transfer of the coupling data between source and target model components. As a new core technology for the software, the fully parallel search algorithm of OASIS4 is described in detail. First benchmark results are discussed with simple test configurations to demonstrate the efficiency and scalability of the software when applied to Earth system model components. Typically the compute time needed to perform the search is in the order of a few seconds and is only weakly dependant on the grid size.

scholarly journals ESM-Tools Version 5.0: A modular infrastructure for stand-alone and coupled Earth System Modelling (ESM)

2021 ◽  
Author(s):  
Dirk Barbi ◽  
Miguel Andrés-Martínez ◽  
Deniz Ural ◽  
Luisa Cristini ◽  
Paul Gierz ◽  
...  
Keyword(s):  
Land Surface ◽  
Ice Sheet ◽  
Multiple Models ◽  
System Modelling ◽  
Specific Information ◽  
Climate Modelling ◽  
Earth System ◽  
Wide Range ◽  
Ocean Atmosphere ◽  
Earth System Modelling

&lt;p&gt;During the last two decades, modern societies have gradually understood the urge to tackle the climate change challenge, and consequently, a growing number of national and international initiatives have been launched with the aim of better understanding the Earth System. In this context, Earth System Modelling (ESM) has rapidly expanded, leading to a large number of research groups targeting the many components of the system at different scales and with different levels of interactions between components. This has led to the development of increasing number of models, couplings, versions tuned to address different scales or scenarios, and model-specific compilation and operating procedures. This operational complexity makes the implementation of multiple models excessively time consuming especially for less experienced modellers.&lt;/p&gt;&lt;p&gt;ESM-Tools is an open-source modular software written in Python, aimed to overcome many of the difficulties associated to the operation of ESMs. ESM-Tools allows for downloading, compiling and running a wide range of ESM models and coupled setups in the most important HPC facilities available in Germany. It currently supports multiple models for ocean, atmosphere, biochemistry, ice sheet, isostatic adjustment, hydrology, and land-surface, and six ocean-atmosphere and two ice-sheet-ocean-atmosphere coupled setups, through two couplers (included modularly through ESM-Interface). The tools are coded in Python while all the component and coupling information is contained in easy-to-read YAML files. The front-end user is required to provide only a short script written in YAML format, containing the experiment specific definitions. This user-friendly interface makes ESM-Tools a convenient software for training and educational purposes. Simultaneously, its modularity and the separation between the component-specific information and tool scripts facilitates the implementation and maintenance of new components, couplings and versions. ESM-Tools team of scientific programmers provides also user support, workshops and detailed documentation. The ESM-Tools were developed within the framework of the project Advance Earth System Model Capacity, supported by Helmholtz Association and has become one of the main pillars of the German infrastructure for Climate Modelling.&lt;/p&gt;

Modular AWI-CM: An Earth System Model (ESM) prototype using the esm-interface library for a modular ESM coupling approach

2020 ◽  
Author(s):  
Nadine Wieters ◽  
Dirk Barbi ◽  
Luisa Cristini
Keyword(s):  
Climate Model ◽  
General Idea ◽  
Second Step ◽  
System Modelling ◽  
Earth System ◽  
First Results ◽  
Coupling Approach ◽  
Domain Models ◽  
Model Components ◽  
Earth System Models

&lt;p&gt;Earth System Models (ESMs) are composed of different components, including submodels as well as whole domain models. Within such an ESM, these model components need to exchange information to account for the interactions between the different compartments. This exchange of data is the purpose of a &amp;#8220;model coupler&amp;#8221;.&lt;/p&gt;&lt;p&gt;Within the Advanced Earth System Modelling Capacity (ESM) project, a goal is to develop a modular framework that allows for a flexible ESM configuration. One approach is to implement purpose build model couplers in a more modular way.&lt;/p&gt;&lt;p&gt;For this purpose, we developed the esm-interface library, in consideration of the following objectives: (i) To obtain a more modular ESM, that allows model components and model couplers to be exchangeable; and (ii) to account for a more flexible coupling configuration of an ESM setup.&lt;/p&gt;&lt;p&gt;As a first application of the esm-interface library, we implemented it into the AWI Climate Model (AWI-CM) [Sidorenko et al., 2015] as an interface between the model components and the model coupler (OASIS3-MCT; Valcke [2013]). In a second step, we extended the esm-interface library for a second coupler (YAC; Hanke et al. [2016]).&lt;/p&gt;&lt;p&gt;In this presentation, we will discuss the general idea of the esm-interface library, it&amp;#8217;s implementation in an ESM setup and show first results from the first modular prototype of AWI-CM.&lt;/p&gt;

scholarly journals Coupling technologies for Earth System Modelling

2012 ◽  
Vol 5 (3) ◽  
pp. 1987-2006 ◽  
Author(s):  
S. Valcke ◽  
V. Balaji ◽  
A. Craig ◽  
C. DeLuca ◽  
R. Dunlap ◽  
...  
Keyword(s):  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Common Features ◽  
The Earth ◽  
Future Challenges ◽  
Computing Platforms ◽  
Design Characteristics ◽  
Earth System Modelling

Abstract. This paper presents a review of the software currently used in climate modelling in general and in CMIP5 in particular to couple the numerical codes representing the different components of the Earth system. The coupling technologies presented show common features, such as the ability to communicate and regrid data, but also offer different functions and implementations. Design characteristics of the different approaches are discussed as well as future challenges arising from the increasing complexity of scientific problems and computing platforms.

scholarly journals ESM-Tools Version 4.0: A modular infrastructure for stand-alone and coupled Earth System Modelling (ESM)

2020 ◽  
Author(s):  
Dirk Barbi ◽  
Nadine Wieters ◽  
Paul Gierz ◽  
Fatemeh Chegini ◽  
Sara Khosravi ◽  
...  
Keyword(s):  
Land Surface ◽  
High Performance ◽  
Surface Model ◽  
System Modelling ◽  
Climate Modelling ◽  
Earth System ◽  
Coupled Models ◽  
Wide Range ◽  
Runtime Infrastructure ◽  
Earth System Modelling

Abstract. Earth system and climate modelling involves the simulation of processes on a wide range of scales and within and across various components of the Earth system. In practice, component models are often developed independently by different research groups and then combined using a dedicated coupling software. This procedure not only leads to a strongly growing number of available versions of model components and coupled setups but also to model- and system-dependent ways of obtaining and operating them. Therefore, implementing these Earth System Models (ESMs) can be challenging and extremely time-consuming, especially for less experienced modellers, or scientists aiming to use different ESMs as in the case of inter-comparison projects. To assist researchers and modellers by reducing avoidable complexity, we developed the ESM-Tools software, which provides a standard way for downloading, configuring, compiling, running and monitoring different models - coupled ESMs and stand-alone models alike - on a variety of High-Performance Computing (HPC) systems. (The ESM-Tools are equally applicable and helpful for stand-alone as for coupled models. In fact, the ESM-Tools are used as standard compile and runtime infrastructure for FESOM2, and currently also applied for ECHAM and ICON standalone simulations. As coupled ESMs are technically the more challenging tasks, we will focus on coupled setups, always implying that stand-alone models can benefit in the same way.) With the ESM-Tools, the user is only required to provide a short script consisting of only the experiment specific definitions, while the software executes all the phases of a simulation in the correct order. The software, which is well documented and easy to install and use, currently supports four ocean models, three atmosphere models, two biogeochemistry models, an ice sheet model, an isostatic adjustment model, a hydrology model and a land-surface model. ESM-Tools has been entirely re-coded in a high-level programming language (Python) and provides researchers with an even more user-friendly interface for Earth system modelling lately. The ESM-Tools were developed within the framework of the project Advanced Earth System Model Capacity, supported by the Helmholtz Association.

scholarly journals Modular System for Shelves and Coasts (MOSSCO v1.0) – a flexible and multi-component framework for coupled coastal ocean ecosystem modelling

2018 ◽  
Vol 11 (3) ◽  
pp. 915-935 ◽  
Author(s):  
Carsten Lemmen ◽  
Richard Hofmeister ◽  
Knut Klingbeil ◽  
M. Hassan Nasermoaddeli ◽  
Onur Kerimoglu ◽  
...  
Keyword(s):  
System Modeling ◽  
Coastal Ocean ◽  
Model Systems ◽  
Modular System ◽  
System Modelling ◽  
Earth System ◽  
Modeling Framework ◽  
Coastal System ◽  
Biogeochemical Models ◽  
Process Coupling

Abstract. Shelf and coastal sea processes extend from the atmosphere through the water column and into the seabed. These processes reflect intimate interactions between physical, chemical, and biological states on multiple scales. As a consequence, coastal system modelling requires a high and flexible degree of process and domain integration; this has so far hardly been achieved by current model systems. The lack of modularity and flexibility in integrated models hinders the exchange of data and model components and has historically imposed the supremacy of specific physical driver models. We present the Modular System for Shelves and Coasts (MOSSCO; http://www.mossco.de), a novel domain and process coupling system tailored but not limited to the coupling challenges of and applications in the coastal ocean. MOSSCO builds on the Earth System Modeling Framework (ESMF) and on the Framework for Aquatic Biogeochemical Models (FABM). It goes beyond existing technologies by creating a unique level of modularity in both domain and process coupling, including a clear separation of component and basic model interfaces, flexible scheduling of several tens of models, and facilitation of iterative development at the lab and the station and on the coastal ocean scale. MOSSCO is rich in metadata and its concepts are also applicable outside the coastal domain. For coastal modelling, it contains dozens of example coupling configurations and tested set-ups for coupled applications. Thus, MOSSCO addresses the technology needs of a growing marine coastal Earth system community that encompasses very different disciplines, numerical tools, and research questions.

scholarly journals Implementation of the Community Earth System Model (CESM) version 1.2.1 as a new base model into version 2.50 of the MESSy framework

2016 ◽  
Vol 9 (1) ◽  
pp. 125-135 ◽  
Author(s):  
A. J. G. Baumgaertner ◽  
P. Jöckel ◽  
A. Kerkweg ◽  
R. Sander ◽  
H. Tost
Keyword(s):  
Atmospheric Chemistry ◽  
Atmospheric Model ◽  
The United States ◽  
Earth System Model ◽  
System Model ◽  
Climate Modelling ◽  
Earth System ◽  
Spectral Transform ◽  
Community Earth System Model ◽  
Component Models

Abstract. The Community Earth System Model (CESM1), maintained by the United States National Centre for Atmospheric Research (NCAR) is connected with the Modular Earth Submodel System (MESSy). For the MESSy user community, this offers many new possibilities. The option to use the Community Atmosphere Model (CAM) atmospheric dynamical cores, especially the state-of-the-art spectral element (SE) core, as an alternative to the ECHAM5 spectral transform dynamical core will provide scientific and computational advances for atmospheric chemistry and climate modelling with MESSy. The well-established finite volume core from CESM1(CAM) is also made available. This offers the possibility to compare three different atmospheric dynamical cores within MESSy. Additionally, the CESM1 land, river, sea ice, glaciers and ocean component models can be used in CESM1/MESSy simulations, allowing the use of MESSy as a comprehensive Earth system model (ESM). For CESM1/MESSy set-ups, the MESSy process and diagnostic submodels for atmospheric physics and chemistry are used together with one of the CESM1(CAM) dynamical cores; the generic (infrastructure) submodels support the atmospheric model component. The other CESM1 component models, as well as the coupling between them, use the original CESM1 infrastructure code and libraries; moreover, in future developments these can also be replaced by the MESSy framework. Here, we describe the structure and capabilities of CESM1/MESSy, document the code changes in CESM1 and MESSy, and introduce several simulations as example applications of the system. The Supplements provide further comparisons with the ECHAM5/MESSy atmospheric chemistry (EMAC) model and document the technical aspects of the connection in detail.

Sign in / Sign up

Export Citation Format

Share Document