Embarrassingly Parallel GPU Based Matrix Inversion Algorithm for Big Climate Data Assimilation

2018 ◽  
Vol 10 (1) ◽  
pp. 71-92 ◽  
Author(s):  
M. Varalakshmi ◽  
Amit Parashuram Kesarkar ◽  
Daphne Lopez
Keyword(s):  
Data Assimilation ◽  
Direct Method ◽  
Weather Prediction ◽  
Matrix Inversion ◽  
Climate Data ◽  
Inversion Algorithm ◽  
Test Environment ◽  
Multiple Gpus ◽  
Resolution Model ◽  
Speed Up

Attempts to harness the big climate data that come from high-resolution model output and advanced sensors to provide more accurate and rapidly-updated weather prediction, call for innovations in the existing data assimilation systems. Matrix inversion is a key operation in a majority of data assimilation techniques. Hence, this article presents out-of-core CUDA implementation of an iterative method of matrix inversion. The results show significant speed up for even square matrices of size 1024 X 1024 and more, without sacrificing the accuracy of the results. In a similar test environment, the comparison of this approach with a direct method such as the Gauss-Jordan approach, modified to process large matrices that cannot be processed directly within a single kernel call shows that the former is twice as efficient as the latter. This acceleration is attributed to the division-free design and the embarrassingly parallel nature of every sub-task of the algorithm. The parallel algorithm has been designed to be highly scalable when implemented with multiple GPUs for handling large matrices.

An Enhanced Multiple Linear Regression Model for Seasonal Rainfall Prediction

2020 ◽  
Vol 10 (4) ◽  
pp. 473-483
Author(s):  
Pundra Chandra Shaker Reddy ◽  
Alladi Sureshbabu
Keyword(s):  
Linear Regression ◽  
Regression Model ◽  
Multiple Linear Regression ◽  
Linear Regression Model ◽  
Weather Prediction ◽  
Big Data Analytics ◽  
Multiple Linear Regression Model ◽  
Seasonal Rainfall ◽  
Climate Data ◽  
Rainfall Forecasting

Aims & Background: India is a country which has exemplary climate circumstances comprising of different seasons and topographical conditions like high temperatures, cold atmosphere, and drought, heavy rainfall seasonal wise. These utmost varieties in climate make us exact weather prediction is a challenging task. Majority people of the country depend on agriculture. Farmers require climate information to decide the planting. Weather prediction turns into an orientation in farming sector to deciding the start of the planting season and furthermore quality and amount of their harvesting. One of the variables are influencing agriculture is rainfall. Objectives & Methods: The main goal of this project is early and proper rainfall forecasting, that helpful to people who live in regions which are inclined natural calamities such as floods and it helps agriculturists for decision making in their crop and water management using big data analytics which produces high in terms of profit and production for farmers. In this project, we proposed an advanced automated framework called Enhanced Multiple Linear Regression Model (EMLRM) with MapReduce algorithm and Hadoop file system. We used climate data from IMD (Indian Metrological Department, Hyderabad) in 1901 to 2002 period. Results: Our experimental outcomes demonstrate that the proposed model forecasting the rainfall with better accuracy compared with other existing models. Conclusion: The results of the analysis will help the farmers to adopt effective modeling approach by anticipating long-term seasonal rainfall.

scholarly journals Use of Microwave Radiances from Metop-C and Fengyun-3 C/D Satellites for a Northern European Limited-area Data Assimilation System

2021 ◽  
Author(s):  
Magnus Lindskog ◽  
Adam Dybbroe ◽  
Roger Randriamampianina
Keyword(s):  
Data Assimilation ◽  
Positive Impact ◽  
Weather Prediction ◽  
Implementation Process ◽  
Satellite System ◽  
Limited Area ◽  
Global Navigation Satellite ◽  
Microwave Radiances ◽  
Data Assimilation System ◽  
Assimilation System

AbstractMetCoOp is a Nordic collaboration on operational Numerical Weather Prediction based on a common limited-area km-scale ensemble system. The initial states are produced using a 3-dimensional variational data assimilation scheme utilizing a large amount of observations from conventional in-situ measurements, weather radars, global navigation satellite system, advanced scatterometer data and satellite radiances from various satellite platforms. A version of the forecasting system which is aimed for future operations has been prepared for an enhanced assimilation of microwave radiances. This enhanced data assimilation system will use radiances from the Microwave Humidity Sounder, the Advanced Microwave Sounding Unit-A and the Micro-Wave Humidity Sounder-2 instruments on-board the Metop-C and Fengyun-3 C/D polar orbiting satellites. The implementation process includes channel selection, set-up of an adaptive bias correction procedure, and careful monitoring of data usage and quality control of observations. The benefit of the additional microwave observations in terms of data coverage and impact on analyses, as derived using the degree of freedom of signal approach, is demonstrated. A positive impact on forecast quality is shown, and the effect on the precipitation for a case study is examined. Finally, the role of enhanced data assimilation techniques and adaptions towards nowcasting are discussed.

scholarly journals Evaluation of Operational Monsoon Moisture Surveillance and Severe Weather Prediction Utilizing COSMIC-2/FORMOSAT-7 Radio Occultation Observations

2021 ◽  
Vol 13 (15) ◽  
pp. 2979
Author(s):  
Yu-Chun Chen ◽  
Chih-Chien Tsai ◽  
Yi-Chao Wu ◽  
An-Hsiang Wang ◽  
Chieh-Ju Wang ◽  
...  
Keyword(s):  
Numerical Model ◽  
Data Assimilation ◽  
Water Resource Management ◽  
Radio Occultation ◽  
Positive Impact ◽  
Weather Prediction ◽  
Severe Weather ◽  
Model Experiments ◽  
Typhoon Structure ◽  
Numerical Model Experiments

Operational monsoon moisture surveillance and severe weather prediction is essential for timely water resource management and disaster risk reduction. For these purposes, this study suggests a moisture indicator using the COSMIC-2/FORMOSAT-7 radio occultation (RO) observations and evaluates numerical model experiments with RO data assimilation. The RO data quality is validated by a comparison between sampled RO profiles and nearby radiosonde profiles around Taiwan prior to the experiments. The suggested moisture indicator accurately monitors daily moisture variations in the South China Sea and the Bay of Bengal throughout the 2020 monsoon rainy season. For the numerical model experiments, the statistics of 152 moisture and rainfall forecasts for the 2020 Meiyu season in Taiwan show a neutral to slightly positive impact brought by RO data assimilation. A forecast sample with the most significant improvement reveals that both thermodynamic and dynamic fields are appropriately adjusted by model integration posterior to data assimilation. The statistics of 17 track forecasts for typhoon Hagupit (2020) also show the positive effect of RO data assimilation. A forecast sample reveals that the member with RO data assimilation simulates better typhoon structure and intensity than the member without, and the effect can be larger and faster via multi-cycle RO data assimilation.

scholarly journals Fengyun Meteorological Satellite Products for Earth System Science Applications

2021 ◽  
Author(s):  
Di Xian ◽  
Peng Zhang ◽  
Ling Gao ◽  
Ruijing Sun ◽  
Haizhen Zhang ◽  
...  
Keyword(s):  
Data Sharing ◽  
Satellite Data ◽  
Prediction Models ◽  
Weather Forecasting ◽  
Numerical Models ◽  
Weather Prediction ◽  
Vegetation Indices ◽  
Open Data ◽  
Earth System ◽  
Inversion Algorithm

AbstractFollowing the progress of satellite data assimilation in the 1990s, the combination of meteorological satellites and numerical models has changed the way scientists understand the earth. With the evolution of numerical weather prediction models and earth system models, meteorological satellites will play a more important role in earth sciences in the future. As part of the space-based infrastructure, the Fengyun (FY) meteorological satellites have contributed to earth science sustainability studies through an open data policy and stable data quality since the first launch of the FY-1A satellite in 1988. The capability of earth system monitoring was greatly enhanced after the second-generation polar orbiting FY-3 satellites and geostationary orbiting FY-4 satellites were developed. Meanwhile, the quality of the products generated from the FY-3 and FY-4 satellites is comparable to the well-known MODIS products. FY satellite data has been utilized broadly in weather forecasting, climate and climate change investigations, environmental disaster monitoring, etc. This article reviews the instruments mounted on the FY satellites. Sensor-dependent level 1 products (radiance data) and inversion algorithm-dependent level 2 products (geophysical parameters) are introduced. As an example, some typical geophysical parameters, such as wildfires, lightning, vegetation indices, aerosol products, soil moisture, and precipitation estimation have been demonstrated and validated by in-situ observations and other well-known satellite products. To help users access the FY products, a set of data sharing systems has been developed and operated. The newly developed data sharing system based on cloud technology has been illustrated to improve the efficiency of data delivery.

scholarly journals Evaluation of WRF-DART (ARW v3.9.1.1 and DART Manhattan release) multiphase cloud water path assimilation for short-term solar irradiance forecasting in a tropical environment

2019 ◽  
Vol 12 (9) ◽  
pp. 3939-3954
Author(s):  
Frederik Kurzrock ◽  
Hannah Nguyen ◽  
Jerome Sauer ◽  
Fabrice Chane Ming ◽  
Sylvain Cros ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Cloud Water ◽  
Specific Humidity ◽  
Reunion Island ◽  
Short Term ◽  
Water Path ◽  
Réunion Island ◽  
Cloud Water Path ◽  
The Impact

Abstract. Numerical weather prediction models tend to underestimate cloud presence and therefore often overestimate global horizontal irradiance (GHI). The assimilation of cloud water path (CWP) retrievals from geostationary satellites using an ensemble Kalman filter (EnKF) led to improved short-term GHI forecasts of the Weather Research and Forecasting (WRF) model in midlatitudes in case studies. An evaluation of the method under tropical conditions and a quantification of this improvement for study periods of more than a few days are still missing. This paper focuses on the assimilation of CWP retrievals in three phases (ice, supercooled, and liquid) in a 6-hourly cycling procedure and on the impact of this method on short-term forecasts of GHI for Réunion Island, a tropical island in the southwest Indian Ocean. The multilayer gridded cloud properties of NASA Langley's Satellite ClOud and Radiation Property retrieval System (SatCORPS) are assimilated using the EnKF of the Data Assimilation Research Testbed (DART) Manhattan release (revision 12002) and the advanced research WRF (ARW) v3.9.1.1. The ability of the method to improve cloud analyses and GHI forecasts is demonstrated, and a comparison using independent radiosoundings shows a reduction of specific humidity bias in the WRF analyses, especially in the low and middle troposphere. Ground-based GHI observations at 12 sites on Réunion Island are used to quantify the impact of CWP DA. Over a total of 44 d during austral summertime, when averaged over all sites, CWP data assimilation has a positive impact on GHI forecasts for all lead times between 5 and 14 h. Root mean square error and mean absolute error are reduced by 4 % and 3 %, respectively.

scholarly journals On the Impact of Unmanned Aerial System Observations on Numerical Weather Prediction in the Coastal Zone

2018 ◽  
Vol 146 (2) ◽  
pp. 599-622 ◽  
Author(s):  
David D. Flagg ◽  
James D. Doyle ◽  
Teddy R. Holt ◽  
Daniel P. Tyndall ◽  
Clark M. Amerault ◽  
...  
Keyword(s):  
Boundary Layer ◽  
Data Assimilation ◽  
Numerical Weather Prediction ◽  
Research Laboratory ◽  
Weather Prediction ◽  
Ocean Model ◽  
Naval Research ◽  
Unmanned Aerial System ◽  
Numerical Weather ◽  
The Impact

Abstract The Trident Warrior observational field campaign conducted off the U.S. mid-Atlantic coast in July 2013 included the deployment of an unmanned aerial system (UAS) with several payloads on board for atmospheric and oceanic observation. These UAS observations, spanning seven flights over 5 days in the lowest 1550 m above mean sea level, were assimilated into a three-dimensional variational data assimilation (DA) system [the Naval Research Laboratory Atmospheric Variational Data Assimilation System (NAVDAS)] used to generate analyses for a numerical weather prediction model [the Coupled Ocean–Atmosphere Mesoscale Prediction System (COAMPS)] with a coupled ocean model [the Naval Research Laboratory Navy Coastal Ocean Model (NCOM)]. The impact of the assimilated UAS observations on short-term atmospheric prediction performance is evaluated and quantified. Observations collected from 50 radiosonde launches during the campaign adjacent to the UAS flight paths serve as model forecast verification. Experiments reveal a substantial reduction of model bias in forecast temperature and moisture profiles consistently throughout the campaign period due to the assimilation of UAS observations. The model error reduction is most substantial in the vicinity of the inversion at the top of the model-estimated boundary layer. Investigations reveal a consistent improvement to prediction of the vertical position, strength, and depth of the boundary layer inversion. The relative impact of UAS observations is explored further with experiments of systematic denial of data streams from the NAVDAS DA system and removal of individual measurement sources on the UAS platform.

Variational Assimilation of Cloud Liquid/Ice Water Path and Its Impact on NWP

2015 ◽  
Vol 54 (8) ◽  
pp. 1809-1825 ◽  
Author(s):  
Yaodeng Chen ◽  
Hongli Wang ◽  
Jinzhong Min ◽  
Xiang-Yu Huang ◽  
Patrick Minnis ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Liquid Water ◽  
Weather Prediction ◽  
Wrf Model ◽  
Liquid Water Path ◽  
Cloud Liquid Water ◽  
Water Path ◽  
Ice Water Path ◽  
Ice Water ◽  
The Impact

AbstractAnalysis of the cloud components in numerical weather prediction models using advanced data assimilation techniques has been a prime topic in recent years. In this research, the variational data assimilation (DA) system for the Weather Research and Forecasting (WRF) Model (WRFDA) is further developed to assimilate satellite cloud products that will produce the cloud liquid water and ice water analysis. Observation operators for the cloud liquid water path and cloud ice water path are developed and incorporated into the WRFDA system. The updated system is tested by assimilating cloud liquid water path and cloud ice water path observations from Global Geostationary Gridded Cloud Products at NASA. To assess the impact of cloud liquid/ice water path data assimilation on short-term regional numerical weather prediction (NWP), 3-hourly cycling data assimilation and forecast experiments with and without the use of the cloud liquid/ice water paths are conducted. It is shown that assimilating cloud liquid/ice water paths increases the accuracy of temperature, humidity, and wind analyses at model levels between 300 and 150 hPa after 5 cycles (15 h). It is also shown that assimilating cloud liquid/ice water paths significantly reduces forecast errors in temperature and wind at model levels between 300 and 150 hPa. The precipitation forecast skills are improved as well. One reason that leads to the improved analysis and forecast is that the 3-hourly rapid update cycle carries over the impact of cloud information from the previous cycles spun up by the WRF Model.

Parallelization Strategies for the GPS Radio Occultation Data Assimilation with a Nonlocal Operator in the Weather Research and Forecasting Model

2014 ◽  
Vol 31 (9) ◽  
pp. 2008-2014 ◽  
Author(s):  
Xin Zhang ◽  
Ying-Hwa Kuo ◽  
Shu-Ya Chen ◽  
Xiang-Yu Huang ◽  
Ling-Feng Hsiao
Keyword(s):  
Data Assimilation ◽  
Radio Occultation ◽  
Weather Prediction ◽  
Weather Research And Forecasting ◽  
Forecasting Model ◽  
Observation Operator ◽  
Gps Radio Occultation ◽  
Phase Observation ◽  
Excess Phase ◽  
Weather Research

Abstract The nonlocal excess phase observation operator for assimilating the global positioning system (GPS) radio occultation (RO) sounding data has been proven by some research papers to produce significantly better analyses for numerical weather prediction (NWP) compared to the local refractivity observation operator. However, the high computational cost and the difficulties in parallelization associated with the nonlocal GPS RO operator deter its application in research and operational NWP practices. In this article, two strategies are designed and implemented in the data assimilation system for the Weather Research and Forecasting Model to demonstrate the capability of parallel assimilation of GPS RO profiles with the nonlocal excess phase observation operator. In particular, to solve the parallel load imbalance problem due to the uneven geographic distribution of the GPS RO observations, round-robin scheduling is adopted to distribute GPS RO observations among the processing cores to balance the workload. The wall clock time required to complete a five-iteration minimization on a demonstration Antarctic case with 106 GPS RO observations is reduced from more than 3.5 h with a single processing core to 2.5 min with 106 processing cores. These strategies present the possibility of application of the nonlocal GPS RO excess phase observation operator in operational data assimilation systems with a cutoff time limit.

Spurious Grid-Scale Precipitation in the North American Regional Reanalysis

2007 ◽  
Vol 135 (6) ◽  
pp. 2168-2184 ◽  
Author(s):  
Gregory L. West ◽  
W. James Steenburgh ◽  
William Y. Y. Cheng
Keyword(s):  
North American ◽  
Prediction Models ◽  
Weather Prediction ◽  
Regional Climate ◽  
Potential Temperature ◽  
Climate Data ◽  
The North ◽  
North American Regional Reanalysis ◽  
Upper Level ◽  
Regional Reanalysis

Abstract Spurious grid-scale precipitation (SGSP) occurs in many mesoscale numerical weather prediction models when the simulated atmosphere becomes convectively unstable and the convective parameterization fails to relieve the instability. Case studies presented in this paper illustrate that SGSP events are also found in the North American Regional Reanalysis (NARR) and are accompanied by excessive maxima in grid-scale precipitation, vertical velocity, moisture variables (e.g., relative humidity and precipitable water), mid- and upper-level equivalent potential temperature, and mid- and upper-level absolute vorticity. SGSP events in environments favorable for high-based convection can also feature low-level cold pools and sea level pressure maxima. Prior to 2003, retrospectively generated NARR analyses feature an average of approximately 370 SGSP events annually. Beginning in 2003, however, NARR analyses are generated in near–real time by the Regional Climate Data Assimilation System (R-CDAS), which is identical to the retrospective NARR analysis system except for the input precipitation and ice cover datasets. Analyses produced by the R-CDAS feature a substantially larger number of SGSP events with more than 4000 occurring in the original 2003 analyses. An oceanic precipitation data processing error, which resulted in a reprocessing of NARR analyses from 2003 to 2005, only partially explains this increase since the reprocessed analyses still produce approximately 2000 SGSP events annually. These results suggest that many NARR SGSP events are not produced by shortcomings in the underlying Eta Model, but by the specification of anomalous latent heating when there is a strong mismatch between modeled and assimilated precipitation. NARR users should ensure that they are using the reprocessed NARR analyses from 2003 to 2005 and consider the possible influence of SGSP on their findings, particularly after the transition to the R-CDAS.

scholarly journals Implementation of a 3-D-Var system for atmospheric profiling data assimilation into the RAMS model: initial results

2013 ◽  
Vol 6 (2) ◽  
pp. 3581-3610
Author(s):  
S. Federico
Keyword(s):  
Relative Humidity ◽  
Data Assimilation ◽  
Initial Conditions ◽  
Weather Prediction ◽  
Wind Profiler ◽  
Short Term ◽  
Recursive Filters ◽  
Data Assimilation System ◽  
Rams Model ◽  
Assimilation System

Abstract. This paper presents the current status of development of a three-dimensional variational data assimilation system. The system can be used with different numerical weather prediction models, but it is mainly designed to be coupled with the Regional Atmospheric Modelling System (RAMS). Analyses are given for the following parameters: zonal and meridional wind components, temperature, relative humidity, and geopotential height. Important features of the data assimilation system are the use of incremental formulation of the cost-function, and the use of an analysis space represented by recursive filters and eigenmodes of the vertical background error matrix. This matrix and the length-scale of the recursive filters are estimated by the National Meteorological Center (NMC) method. The data assimilation and forecasting system is applied to the real context of atmospheric profiling data assimilation, and in particular to the short-term wind prediction. The analyses are produced at 20 km horizontal resolution over central Europe and extend over the whole troposphere. Assimilated data are vertical soundings of wind, temperature, and relative humidity from radiosondes, and wind measurements of the European wind profiler network. Results show the validity of the analysis solutions because they are closer to the observations (lower RMSE) compared to the background (higher RMSE), and the differences of the RMSEs are consistent with the data assimilation settings. To quantify the impact of improved initial conditions on the short-term forecast, the analyses are used as initial conditions of a three-hours forecast of the RAMS model. In particular two sets of forecasts are produced: (a) the first uses the ECMWF analysis/forecast cycle as initial and boundary conditions; (b) the second uses the analyses produced by the 3-D-Var scheme as initial conditions, then is driven by the ECMWF forecast. The improvement is quantified by considering the horizontal components of the wind, which are measured at a-synoptic times by the European wind profiler network. The results show that the RMSE is effectively reduced at the short range (1–2 h). The results are in agreement with the set-up of the numerical experiment.

Sign in / Sign up

Export Citation Format

Share Document