In Situ and In Transit Computing for Large Scale Geoscientific Simulation

2020 ◽  
Author(s):  
Sebastian Friedemann ◽  
Bruno Raffin ◽  
Basile Hector ◽  
Jean-Martial Cohard
Keyword(s):  
Data Assimilation ◽  
High Performance ◽  
Large Scale ◽  
Transfer Model ◽  
Simulation Data ◽  
Computing Platform ◽  
Continental Scale ◽  
Physically Based ◽  
In Transit

<p>In situ and in transit computing is an effective way to place postprocessing and preprocessing tasks for large scale simulations on the high performance computing platform. The resulting proximity between the execution of preprocessing, simulation and postprocessing permits to lower I/O by bypassing slow and energy inefficient persistent storages. This permits to scale workflows consisting of heterogeneous components such as simulation, data analysis and visualization, to modern massively parallel high performance platforms. Reordering the workflow components gives a manifold of new advanced data processing possibilities for research. Thus in situ and in transit computing are vital for advances in the domain of geoscientific simulation which relies on the increasing amount of sensor and simulation data available.</p><p>In this talk, different in situ and in transit workflows, especially those that are useful in the field of geoscientific simulation, are discussed. Furthermore our experiences augmenting ParFlow-CLM, a physically based, state-of-the-art, fully coupled water transfer model for the critical zone, with FlowVR, an in situ framework with a strict component paradigm, are presented.<br>This allows shadowed in situ file writing, in situ online steering and in situ visualization.</p><p>In situ frameworks further can be coupled to data assimilation tools.<br>In the on going EoCoE-II we propose to embed data assimilation codes into an in transit computing environment. This is expected to enable ensemble based data assimilation on continental scale hydrological simulations with multiple thousands of ensemble members.</p>

scholarly journals A Physically Based Empirical Localization Method for Assimilating Synthetic SWOT Observations of a Continental-Scale River: A Case Study in the Congo Basin

Water ◽  
2019 ◽  
Vol 11 (4) ◽  
pp. 829 ◽  
Author(s):  
Revel ◽  
Ikeshima ◽  
Yamazaki ◽  
Kanae
Keyword(s):  
Data Assimilation ◽  
Hydrodynamic Modeling ◽  
Congo Basin ◽  
Localization Method ◽  
Continental Scale ◽  
Error Covariance ◽  
Physically Based ◽  
Local Patch ◽  
In Situ Observations

Water resource management has faced challenges in recent decades due to limited in situ observations and the limitations of hydrodynamic modeling. Data assimilation techniques have been proposed to improve hydrodynamic model outputs of local rivers (river length ≤ 1500 km) using synthetic observations of the future Surface Water and Ocean Topography (SWOT) satellite mission to overcome limited in situ observations and the limitations of hydrodynamic modeling. However, large-scale data assimilation schemes require computationally efficient filtering techniques, such as the Local Ensemble Transformation Kalman Filter (LETKF). Expansion of the assimilation domain to maximize observations is limited by error covariance caused by limited ensemble size in complex river networks, such as the Congo River. Therefore, we tested the LETKF algorithm in a continental-scale river (river length > 1500 km) using a physically based empirical localization method to maximize the observations available while filtering error covariance areas. Physically based empirical local patches were derived separately for each river pixel, considering spatial auto-correlations. An observing system simulation experiment (OSSE) was performed using empirical localization parameters to evaluate the potential of our method for estimating discharge. We found our method could improve discharge estimates considerably without affected from error covariance while fully using the available observations. We compared this experiment using empirical localization parameters with conventional fixed-shape local patches of different sizes. The empirical local patch OSSE showed the lowest normalized root mean square error of discharge for the entire Congo basin. Extending the conventional local patch without considering spatial auto-correlation results in very large errors in LETKF assimilation due to error covariance between small tributaries. The empirical local patch method has the potential to overcome the limitations of conventional local patches for continental-scale rivers using SWOT observations.

Massive Image Treatment System Based on Cloud Computing Platform

2014 ◽  
Vol 687-691 ◽  
pp. 3733-3737
Author(s):  
Dan Wu ◽  
Ming Quan Zhou ◽  
Rong Fang Bie
Keyword(s):  
Image Processing ◽  
Cloud Computing ◽  
High Performance ◽  
Large Scale ◽  
Processing System ◽  
Virtual Space ◽  
Image Processing System ◽  
Computing Platform ◽  
Simulation Calculation ◽  
Computer Resources

Massive image processing technology requires high requirements of processor and memory, and it needs to adopt high performance of processor and the large capacity memory. While the single or single core processing and traditional memory can’t satisfy the need of image processing. This paper introduces the cloud computing function into the massive image processing system. Through the cloud computing function it expands the virtual space of the system, saves computer resources and improves the efficiency of image processing. The system processor uses multi-core DSP parallel processor, and develops visualization parameter setting window and output results using VC software settings. Through simulation calculation we get the image processing speed curve and the system image adaptive curve. It provides the technical reference for the design of large-scale image processing system.

scholarly journals Using data assimilation to optimize pedotransfer functions using large-scale in-situ soil moisture observations

2020 ◽  
Author(s):  
Elizabeth Cooper ◽  
Eleanor Blyth ◽  
Hollie Cooper ◽  
Rich Ellis ◽  
Ewan Pinnington ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Data Assimilation ◽  
Land Surface ◽  
Large Scale ◽  
Pedotransfer Functions ◽  
Surface Model ◽  
Climate Projections ◽  
Land Surface Models ◽  
Surface Models

Abstract. Soil moisture predictions from land surface models are important in hydrological, ecological and meteorological applications. In recent years the availability of wide-area soil-moisture measurements has increased, but few studies have combined model-based soil moisture predictions with in-situ observations beyond the point scale. Here we show that we can markedly improve soil moisture estimates from the JULES land surface model using field scale observations and data assimilation techniques. Rather than directly updating soil moisture estimates towards observed values, we optimize constants in the underlying pedotransfer functions, which relate soil texture to JULES soil physics parameters. In this way we generate a single set of newly calibrated pedotransfer functions based on observations from a number of UK sites with different soil textures. We demonstrate that calibrating a pedotransfer function in this way can improve the performance of land surface models, leading to the potential for better flood, drought and climate projections.

scholarly journals Assimilating in situ and radar altimetry data into a large-scale hydrologic-hydrodynamic model for streamflow forecast in the Amazon

2013 ◽  
Vol 10 (3) ◽  
pp. 2879-2925 ◽  
Author(s):  
R. C. D. Paiva ◽  
W. Collischonn ◽  
M.-P. Bonnet ◽  
L. G. G. de Gonçalves ◽  
S. Calmant ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Hydrodynamic Model ◽  
Stream Flow ◽  
Large Scale ◽  
Water Levels ◽  
Large Rivers ◽  
Altimetry Data ◽  
Radar Altimetry ◽  
Prediction Approach

Abstract. In this work we introduce and evaluate a data assimilation framework for gauged and radar altimetry-based discharge and water levels applied to a large scale hydrologic-hydrodynamic model for stream flow forecasts over the Amazon River basin. We used the process-based hydrological model called MGB-IPH coupled with a river hydrodynamic module using a storage model for floodplains. The Ensemble Kalman Filter technique was used to assimilate information from hundreds of gauging and altimetry stations based on ENVISAT satellite data. Model state variables errors were generated by corrupting precipitation forcing, considering log-normally distributed, time and spatially correlated errors. The EnKF performed well when assimilating in situ discharge, by improving model estimates at the assimilation sites and also transferring information to ungauged rivers reaches. Altimetry data assimilation improves results at a daily basis in terms of water levels and discharges with minor degree, even though radar altimetry data has a low temporal resolution. Sensitivity tests highlighted the importance of the magnitude of the precipitation errors and that of their spatial correlation, while temporal correlation showed to be dispensable. The deterioration of model performance at some unmonitored reaches indicates the need for proper characterization of model errors and spatial localization techniques for hydrological applications. Finally, we evaluated stream flow forecasts for the Amazon basin based on initial conditions produced by the data assimilation scheme and using the ensemble stream flow prediction approach where the model is forced by past meteorological forcings. The resulting forecasts agreed well with the observations and maintained meaningful skill at large rivers even for long lead times, e.g. > 90 days at the Solimões/Amazon main stem. Results encourage the potential of hydrological forecasts at large rivers and/or poorly monitored regions by combining models and remote sensing information.

scholarly journals FREE GLOBAL DSM ASSESSMENT ON LARGE SCALE AREAS EXPLOITING THE POTENTIALITIES OF THE INNOVATIVE GOOGLE EARTH ENGINE PLATFORM

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 627-633 ◽  
Author(s):  
A. Nascetti ◽  
M. Di Rita ◽  
R. Ravanelli ◽  
M. Amicuzi ◽  
S. Esposito ◽  
...  
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Global Scale ◽  
Google Earth ◽  
Observation Data ◽  
Geometric Accuracy ◽  
Computing Platform ◽  
Aster Gdem ◽  
Google Earth Engine ◽  
Earth Observation Data

The high-performance cloud-computing platform Google Earth Engine has been developed for global-scale analysis based on the Earth observation data. In particular, in this work, the geometric accuracy of the two most used nearly-global free DSMs (SRTM and ASTER) has been evaluated on the territories of four American States (Colorado, Michigan, Nevada, Utah) and one Italian Region (Trentino Alto- Adige, Northern Italy) exploiting the potentiality of this platform. These are large areas characterized by different terrain morphology, land covers and slopes. The assessment has been performed using two different reference DSMs: the USGS National Elevation Dataset (NED) and a LiDAR acquisition. The DSMs accuracy has been evaluated through computation of standard statistic parameters, both at global scale (considering the whole State/Region) and in function of the terrain morphology using several slope classes. The geometric accuracy in terms of Standard deviation and NMAD, for SRTM range from 2-3 meters in the first slope class to about 45 meters in the last one, whereas for ASTER, the values range from 5-6 to 30 meters.<br><br> In general, the performed analysis shows a better accuracy for the SRTM in the flat areas whereas the ASTER GDEM is more reliable in the steep areas, where the slopes increase. These preliminary results highlight the GEE potentialities to perform DSM assessment on a global scale.

scholarly journals Physically based retrieval of crop characteristics for improved water use estimates

2009 ◽  
Vol 13 (5) ◽  
pp. 663-674 ◽  
Author(s):  
K. Richter ◽  
W. J. Timmermans
Keyword(s):  
Sustainable Management ◽  
Test Site ◽  
Balance Model ◽  
Radiative Transfer Model ◽  
Irrigated Agriculture ◽  
Transfer Model ◽  
Retrieval Performance ◽  
Area Index ◽  
Physically Based

Abstract. The increasing scarcity of water from local to global scales requires the efficient monitoring of this valuable resource, especially in the context of a sustainable management in irrigated agriculture. In this study, a two-source energy balance model (TSEB) was applied to the Barrax test site. The inputs of leaf area index (LAI) and fractional vegetation cover (fCover) were estimated from CHRIS imagery by using the traditional scaled NDVI and a look-up table (LUT) inversion approach. The LUT was constructed by using the well established SAILH + PROSPECT radiative transfer model. Simulated fluxes were compared with tower measurements and vegetation characteristics were evaluated with in situ LAI and fCover measurements of a range of crops from the SPARC campaign 2004. Results showed a better retrieval performance for the LUT approach for canopy parameters, affecting flux predictions that were related to land use.

scholarly journals Chip on a fiber toward the e-textile computing platform

2020 ◽  
Author(s):  
Sunbin Hwang ◽  
Minji Kang ◽  
Aram Lee ◽  
Sukang Bae ◽  
Seoung-Ki Lee ◽  
...  
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Electronic Device ◽  
Electronic Textiles ◽  
Electronic Components ◽  
Computing Platform ◽  
Large Scale Integration ◽  
Circuit Components ◽  
Scale Integration ◽  
Electronic Textile

Abstract Electronic textiles have been considered one of the desired device platforms due to their dimensional compatibility with fabrics by weaving them with yarn. However, the existing electronic textile platforms are generally composed of only one type of electronic component with a single function on a fiber substrate because of processing challenges. A precise connecting process between each electronic fiber is essential to configure the desired electronic circuits or systems. Here we present a chip on a fiber, a new electronic fiber platform, by introducing large scale integration of electronic device or circuit components onto a one-dimensional microfiber substrate. The electronic components such as transistors, inverters, ring oscillators, and thermocouples were integrated together onto the outer surface of a fiber substrate with precise semiconductor and electrode patterns. Our results show that the electronic components can be integrated on a single fiber with reliable operation. We evaluate the electronic properties of the chip on a fiber as a multifunctional electronic textile platform by testing their switching and data processing, as well as sensing or transducing units for detecting optical/thermal signals. The demonstration of the chip on a fiber suggests significant proof of concepts for realization of high performance with wearable electronic textile systems.

scholarly journals Assimilation of Soil Moisture and Ocean Salinity (SMOS) brightness temperature into a large-scale distributed conceptual hydrological model to improve soil moisture predictions: the Murray–Darling basin in Australia as a test case

2020 ◽  
Vol 24 (10) ◽  
pp. 4793-4812
Author(s):  
Renaud Hostache ◽  
Dominik Rains ◽  
Kaniska Mallick ◽  
Marco Chini ◽  
Ramona Pelich ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Brightness Temperature ◽  
Large Scale ◽  
Root Zone ◽  
Meteorological Model ◽  
Murray Darling Basin ◽  
Ocean Salinity ◽  
Physically Based ◽  
Temperature Observations

Abstract. The main objective of this study is to investigate how brightness temperature observations from satellite microwave sensors may help to reduce errors and uncertainties in soil moisture and evapotranspiration simulations with a large-scale conceptual hydro-meteorological model. In addition, this study aims to investigate whether such a conceptual modelling framework, relying on parameter calibration, can reach the performance level of more complex physically based models for soil moisture simulations at a large scale. We use the ERA-Interim publicly available forcing data set and couple the Community Microwave Emission Modelling (CMEM) platform radiative transfer model with a hydro-meteorological model to enable, therefore, soil moisture, evapotranspiration and brightness temperature simulations over the Murray–Darling basin in Australia. The hydro-meteorological model is configured using recent developments in the SUPERFLEX framework, which enables tailoring the model structure to the specific needs of the application and to data availability and computational requirements. The hydrological model is first calibrated using only a sample of the Soil Moisture and Ocean Salinity (SMOS) brightness temperature observations (2010–2011). Next, SMOS brightness temperature observations are sequentially assimilated into the coupled SUPERFLEX–CMEM model (2010–2015). For this experiment, a local ensemble transform Kalman filter is used. Our empirical results show that the SUPERFLEX–CMEM modelling chain is capable of predicting soil moisture at a performance level similar to that obtained for the same study area and with a quasi-identical experimental set-up using the Community Land Model (CLM) . This shows that a simple model, when calibrated using globally and freely available Earth observation data, can yield performance levels similar to those of a physically based (uncalibrated) model. The correlation between simulated and in situ observed soil moisture ranges from 0.62 to 0.72 for the surface and root zone soil moisture. The assimilation of SMOS brightness temperature observations into the SUPERFLEX–CMEM modelling chain improves the correlation between predicted and in situ observed surface and root zone soil moisture by 0.03 on average, showing improvements similar to those obtained using the CLM land surface model. Moreover, at the same time the assimilation improves the correlation between predicted and in situ observed monthly evapotranspiration by 0.02 on average.

Sign in / Sign up

Export Citation Format

Share Document