Collaborative Project: The problem of bias in defining uncertainty in computationally enabled strategies for data-driven climate model development

Since their first operational application in the 1950s, atmospheric numerical models have become essential tools in weather and climate prediction. As such, they are a constant subject to changes, thanks to advances in computer systems, numerical methods, and the ever increasing knowledge about the atmosphere of Earth. Many of the changes in today's models relate to seemingly unsuspicious modifications, associated with minor code rearrangements, changes in hardware infrastructure, or software upgrades. Such changes are meant to preserve the model formulation, yet the verification of such changes is challenged by the chaotic nature of our atmosphere - any small change, even rounding errors, can have a big impact on individual simulations. Overall this represents a serious challenge to a consistent model development and maintenance framework.Here we propose a new methodology for quantifying and verifying the impacts of minor atmospheric model changes, or its underlying hardware/software system, by using ensemble simulations in combination with a statistical hypothesis test. The methodology can assess effects of model changes on almost any output variable over time, and can also be used with different hypothesis tests.We present first applications of the methodology with the regional weather and climate model COSMO. The changes considered include a major system upgrade of the supercomputer used, the change from double to single precision floating-point representation, changes in the update frequency of the lateral boundary conditions, and tiny changes to selected model parameters. While providing very robust results, the methodology also shows a large sensitivity to more significant model changes, making it a good candidate for an automated tool to guarantee model consistency in the development cycle.

Download Full-text

Using a virtual machine environment for developing, testing, and training for the UM-UKCA composition-climate model, using Unified Model version 10.9 and above

Geoscientific Model Development ◽

10.5194/gmd-11-3647-2018 ◽

2018 ◽

Vol 11 (9) ◽

pp. 3647-3657 ◽

Cited By ~ 1

Author(s):

Nathan Luke Abraham ◽

Alexander T. Archibald ◽

Paul Cresswell ◽

Sam Cusworth ◽

Mohit Dalvi ◽

...

Keyword(s):

Virtual Machine ◽

Climate Model ◽

State Of The Art ◽

Model Development ◽

Unified Model ◽

Model Version ◽

Weather And Climate ◽

Interactive Composition ◽

Comprehensive Test ◽

And Training

Abstract. The Met Office Unified Model (UM) is a state-of-the-art weather and climate model that is used operationally worldwide. UKCA is the chemistry and aerosol sub model of the UM that enables interactive composition and physical atmosphere interactions, but which adds an additional 120 000 lines of code to the model. Ensuring that the UM code and UM-UKCA (the UM running with interactive chemistry and aerosols) is well tested is thus essential. While a comprehensive test harness is in place at the Met Office and partner sites to aid in development, this is not available to many UM users. Recently, the Met Office have made available a virtual machine environment that can be used to run the UM on a desktop or laptop PC. Here we describe the development of a UM-UKCA configuration that is able to run within this virtual machine while only needing 6 GB of memory, before discussing the applications of this system for model development, testing, and training.

Download Full-text

Comparison and assessment of large-scale surface temperature in climate model simulations

Advances in Statistical Climatology Meteorology and Oceanography ◽

10.5194/ascmo-5-67-2019 ◽

2019 ◽

Vol 5 (1) ◽

pp. 67-85

Author(s):

Raquel Barata ◽

Raquel Prado ◽

Bruno Sansó

Keyword(s):

Surface Temperature ◽

Seasonal Changes ◽

Large Scale ◽

Climate Model ◽

Internal Variability ◽

Temperature Variability ◽

Data Driven ◽

Model Simulations ◽

Climate Model Simulations

Abstract. We present a data-driven approach to assess and compare the behavior of large-scale spatial averages of surface temperature in climate model simulations and in observational products. We rely on univariate and multivariate dynamic linear model (DLM) techniques to estimate both long-term and seasonal changes in temperature. The residuals from the DLM analyses capture the internal variability of the climate system and exhibit complex temporal autocorrelation structure. To characterize this internal variability, we explore the structure of these residuals using univariate and multivariate autoregressive (AR) models. As a proof of concept that can easily be extended to other climate models, we apply our approach to one particular climate model (MIROC5). Our results illustrate model versus data differences in both long-term and seasonal changes in temperature. Despite differences in the underlying factors contributing to variability, the different types of simulation yield very similar spectral estimates of internal temperature variability. In general, we find that there is no evidence that the MIROC5 model systematically underestimates the amplitude of observed surface temperature variability on multi-decadal timescales – a finding that has considerable relevance regarding efforts to identify anthropogenic “fingerprints” in observational surface temperature data. Our methodology and results present a novel approach to obtaining data-driven estimates of climate variability for purposes of model evaluation.

Download Full-text

Data-driven multiscale modeling reveals the role of metabolic coupling for the spatio-temporal growth dynamics of yeast colonies

10.1101/344226 ◽

2018 ◽

Author(s):

Jukka Intosalmi ◽

Adrian C. Scott ◽

Michelle Hays ◽

Nicholas Flann ◽

Olli Yli-Harja ◽

...

Keyword(s):

Experimental Data ◽

Time Course ◽

Model Development ◽

Growth Dynamics ◽

Colony Growth ◽

Data Driven ◽

Coarse Grained ◽

Intercellular Signaling ◽

Growth Data ◽

Metabolic Coupling

AbstractMotivationMulticellular entities, such as mammalian tissues or microbial biofilms, typically exhibit complex spatial arrangements that are adapted to their specific functions or environments. These structures result from intercellular signaling as well as from the interaction with the environment that allow cells of the same genotype to differentiate into well-organized communities of diversified cells. Despite its importance, our understanding on how cell–cell and metabolic coupling produce functionally optimized structures is still limited.ResultsHere, we present a data-driven spatial framework to computationally investigate the development of one multicellular structure, yeast colonies. Using experimental growth data from homogeneous liquid media conditions, we develop and parameterize a dynamic cell state and growth model. We then use the resulting model in a coarse-grained spatial model, which we calibrate using experimental time-course data of colony growth. Throughout the model development process, we use state-of-the-art statistical techniques to handle the uncertainty of model structure and parameterization. Further, we validate the model predictions against independent experimental data and illustrate how metabolic coupling plays a central role in colony formation.AvailabilityExperimental data and a computational implementation to reproduce the results are available athttp://research.cs.aalto.fi/csb/software/multiscale/[email protected],[email protected]

Download Full-text

Machine learning in prognosis research

Prognosis Research in Health Care ◽

10.1093/med/9780198796619.003.0017 ◽

2019 ◽

pp. 328-342

Author(s):

Mihaela van der Schaar ◽

Harry Hemingway

Keyword(s):

Machine Learning ◽

Health Outcomes ◽

Prognostic Model ◽

Dynamic Models ◽

Model Development ◽

Data Driven ◽

Complex Data ◽

Pragmatic Approach ◽

Prognosis Research ◽

Research Types

Machine learning offers an alternative to the methods for prognosis research in large and complex datasets and for delivering dynamic models of prognosis. Machine learning foregrounds the capacity to learn from large and complex data about the pathways, predictors, and trajectories of health outcomes in individuals. This reflects wider societal drives for data-driven modelling embedded and automated within powerful computers to analyse large amounts of data. Machine learning derives algorithms that can learn from data and can allow the data full freedom, for example, to follow a pragmatic approach in developing a prognostic model. Rather than choosing factors for model development in advance, machine learning allows the data to reveal which features are important for which predictions. This chapter introduces key machine learning concepts relevant to each of the four prognosis research types, explains where it may enhance prognosis research, and highlights challenges.

Download Full-text

A Data-Driven Approach for Winter Precipitation Classification Using Weather Radar and NWP Data

Atmosphere ◽

10.3390/atmos11070701 ◽

2020 ◽

Vol 11 (7) ◽

pp. 701

Author(s):

Bong-Chul Seo

Keyword(s):

Random Forest ◽

Binary Classification ◽

Weather Prediction ◽

Model Development ◽

Winter Precipitation ◽

Ensemble Classification ◽

Supervised Machine Learning ◽

Data Driven ◽

Support Vector ◽

Data Driven Approach

This study describes a framework that provides qualitative weather information on winter precipitation types using a data-driven approach. The framework incorporates the data retrieved from weather radars and the numerical weather prediction (NWP) model to account for relevant precipitation microphysics. To enable multimodel-based ensemble classification, we selected six supervised machine learning models: k-nearest neighbors, logistic regression, support vector machine, decision tree, random forest, and multi-layer perceptron. Our model training and cross-validation results based on Monte Carlo Simulation (MCS) showed that all the models performed better than our baseline method, which applies two thresholds (surface temperature and atmospheric layer thickness) for binary classification (i.e., rain/snow). Among all six models, random forest presented the best classification results for the basic classes (rain, freezing rain, and snow) and the further refinement of the snow classes (light, moderate, and heavy). Our model evaluation, which uses an independent dataset not associated with model development and learning, led to classification performance consistent with that from the MCS analysis. Based on the visual inspection of the classification maps generated for an individual radar domain, we confirmed the improved classification capability of the developed models (e.g., random forest) compared to the baseline one in representing both spatial variability and continuity.

Download Full-text

Data-driven model development for quality prediction in forming technology

2017 IEEE 15th International Conference on Industrial Informatics (INDIN) ◽

10.1109/indin.2017.8104871 ◽

2017 ◽

Cited By ~ 2

Author(s):

Iris Kirchen ◽

Birgit Vogel-Heuser ◽

Philipp Hildenbrand ◽

Robert Schulte ◽

Manfred Vogel ◽

...

Keyword(s):

Model Development ◽

Data Driven ◽

Quality Prediction ◽

Forming Technology

Download Full-text

DART/CAM: An Ensemble Data Assimilation System for CESM Atmospheric Models

Journal of Climate ◽

10.1175/jcli-d-11-00395.1 ◽

2012 ◽

Vol 25 (18) ◽

pp. 6304-6317 ◽

Cited By ~ 45

Author(s):

Kevin Raeder ◽

Jeffrey L. Anderson ◽

Nancy Collins ◽

Timothy J. Hoar ◽

Jennifer E. Kay ◽

...

Keyword(s):

Data Assimilation ◽

Climate Model ◽

Initial Conditions ◽

Model Development ◽

Arctic Sea Ice ◽

Large Set ◽

State Variables ◽

Ensemble Data Assimilation ◽

Ensemble Data ◽

Atmospheric Forcings

Abstract The Community Atmosphere Model (CAM) has been interfaced to the Data Assimilation Research Testbed (DART), a community facility for ensemble data assimilation. This provides a large set of data assimilation tools for climate model research and development. Aspects of the interface to the Community Earth System Model (CESM) software are discussed and a variety of applications are illustrated, ranging from model development to the production of long series of analyses. CAM output is compared directly to real observations from platforms ranging from radiosondes to global positioning system satellites. Such comparisons use the temporally and spatially heterogeneous analysis error estimates available from the ensemble to provide very specific forecast quality evaluations. The ability to start forecasts from analyses, which were generated by CAM on its native grid and have no foreign model bias, contributed to the detection of a code error involving Arctic sea ice and cloud cover. The potential of parameter estimation is discussed. A CAM ensemble reanalysis has been generated for more than 15 yr. Atmospheric forcings from the reanalysis were required as input to generate an ocean ensemble reanalysis that provided initial conditions for decadal prediction experiments. The software enables rapid experimentation with differing sets of observations and state variables, and the comparison of different models against identical real observations, as illustrated by a comparison of forecasts initialized by interpolated ECMWF analyses and by DART/CAM analyses.

Download Full-text

Process-based climate model development harnessing machine learning: II. model calibration from single column to global

10.1002/essoar.10503845.1 ◽

2020 ◽

Author(s):

Frédéric Hourdin ◽

Danny Williamson ◽

Catherine Rio ◽

Fleur Couvreux ◽

Romain Roehrig ◽

...

Keyword(s):

Machine Learning ◽

Model Calibration ◽

Climate Model ◽

Model Development ◽

Single Column

Download Full-text