Real time weather for autonomous driving and precise road weather forecasts based on floating car data and seamless integration of Ultra-Rapid Data Assimilation and Nowcasting

The growing availability of high resolved meteorological measurements coming from automobiles puts forward the possibility of developing real time weather forecast systems, which appears to be an essential key of autonomous driving enhancement. In this frame, the Fleet Weather Maps (Flotten-Wetter-Karte - FloWKar) project, a joint work of the German Meteorological Service (DWD) and the German car manufacturer AUDI AG, aims to explore how environmental data from sensors of vehicles on Germany&#8217;s roads, respecting data protection regulations, can be used in real time to improve weather forecast, nowcasting and warnings within DWD&#8217;s products. Regarding weather forecasting, an exceptionally fast data assimilation cycle with an update rate of the order of minutes is necessary. However, this cannot be achieved using standard assimilation systems. Hence, an ultra-rapid data assimilation (URDA) method has been developed. The URDA updates only a reduced version of the state variables in an existing model forecast, using different kind of observation data available, only after a standard assimilation cycle and a full model forecast. Moreover, the quality of the meteorological data collected by moving vehicles is vital and therefore a series of quality control and bias correction algorithms has been built for the correction of the raw observations, employing among others artificial intelligence techniques. The first preliminary results of both project partners are promising: the corrected measured variables of the mass-produced vehicle-based sensors match well with the &#8216;ground truth&#8217; and real time maps are produced after the assimilation of the high resolved project data. The improved and detailed model outputs for road weather forecasting are a first necessary step towards the safety on roads especially in the winter conditions and the future autonomous driving.

Download Full-text

Numerical Weather Prediction for Himalayan Complex Terrain: Prospects of Variational Data Assimilation

Journal of Nepal Physical Society ◽

10.3126/jnphyssoc.v3i1.14445 ◽

2016 ◽

Vol 3 (1) ◽

pp. 67

Author(s):

Sangeeta Maharjan ◽

Ram P. Regmi

Keyword(s):

Data Assimilation ◽

Weather Prediction ◽

Weather Forecast ◽

Environmental Research ◽

Data Sets ◽

Observation Data ◽

Ongoing Research ◽

Medium Range Weather Forecast ◽

Research Activities ◽

Wrf Modeling

As part of the ongoing research activities at National Atmospheric Resource and Environmental Research Laboratory (NARERL) to realize high spatial and temporal resolution weather forecasts for Nepal, the Weather Research and Forecasting (WRF) modeling system performance with the National Center for Environmental Protection (NCEP) and National Center for Medium Range Weather Forecast (NCMRWF) initialization global meteorological data sets and the effect of surface observation data assimilation have been examined. The study shows that WRF modeling system reasonably well predicts the diurnal variation of upcoming weather events with both the data sets. The observation data assimilation from entire weather station distributed over the country may lead to the significant improvement in the accuracy and reliability of extended period of forecast. However, upper air observation data assimilation would be necessary to achieve desired precision and reliability of extended weather forecast.Journal of Nepal Physical Society Vol.3(1) 2015: 67-72

Download Full-text

Establishing the Advanced Disaster Reduction Management System by Fusion of Real-Time Disaster Simulation and Big Data Assimilation

Journal of Disaster Research ◽

10.20965/jdr.2016.p0164 ◽

2016 ◽

Vol 11 (2) ◽

pp. 164-174 ◽

Cited By ~ 12

Author(s):

Shunichi Koshimura ◽

Keyword(s):

Big Data ◽

Data Analysis ◽

Data Assimilation ◽

Real Time ◽

Big Data Analysis ◽

Disaster Mitigation ◽

Observation Data ◽

Real Time Simulation ◽

Time Simulation ◽

Analysis Platform

A project titled “Establishing the advanced disaster reduction management system by fusion of real-time disaster simulation and big data assimilation,” was launched as Core Research for Evolutional Science and Technology (CREST) by the Japan Science and Technology Agency (JST). Intended to save as many lives as possible in future national crises involving earthquake and tsunami disasters, the project works on a disaster mitigation system of the big data era, based on cooperation of large-scale, high-resolution, real-time numerical simulations and assimilation of real-time observation data. The world’s most advanced specialists in disaster simulation, disaster management, mathematical science, and information science work together to create the world’s first analysis platform for real-time simulation and big data that effectively processes, analyzes, and assimilates data obtained through various observations. Based on quantitative data, the platform designs proactive measures and supports disaster operations immediately after disaster occurrence. The project was launched in 2014 and is working on the following issues at present.Sophistication and fusion of simulations and damage prediction models using observational big data: Development of a real-time simulation core system that predicts the time evolution of disaster effect by assimilating of location information, fire information, and building collapse information which are obtained from mobile terminals, satellite images, aerial images, and other new observation data in addition to sensing data obtained by the undersea high-density seismic observation network.Latent structure analysis and major disaster scenario creation based on a huge amount of simulation results: Development of an analysis and extraction method for the latent structure of a huge amount of disaster scenarios generated by simulation, and creation of severe scenarios with minimum “unexpectedness” by controlling disaster scenario explosion (an explosive increase in the number of predicted scenarios).Establishment of an earthquake and tsunami disaster mitigation big data analysis platform: Development of an earthquake and tsunami disaster mitigation big data analysis platform that realizes analyses of a huge number of disaster scenarios and increases in speed of data assimilation, and clarifies the requirements for operation of the platform as a disaster mitigation system.The project was launched in 2014 as a 5-year project. It consists of element technology development and system fusion, feasibility study as a next-generation disaster mitigation system (validation with/without introduction of the developed real-time simulation and big data analysis platform) in the affected areas of the Great East Japan Earthquake, and test operations in affected areas of the Tokyo metropolitan earthquake and the Nankai Trough earthquake.

Download Full-text

Refinement of CrIS channel selection for global data assimilation and its impact on the global weather forecast

Weather and Forecasting ◽

10.1175/waf-d-21-0002.1 ◽

2021 ◽

Author(s):

Young-Chan Noh ◽

Hung-Lung Huang ◽

Mitchell D. Goldberg

Keyword(s):

Water Vapor ◽

Data Assimilation ◽

Weather Forecasting ◽

Selection Process ◽

Forecast Error ◽

Weather Forecast ◽

Error Covariance ◽

Environmental Prediction ◽

Cross Track ◽

Global Data

AbstractTo maximize the contribution of the Cross-track Infrared Sounder (CrIS) measurements to the global weather forecasting, we attempt to choose the CrIS channels to be assimilated in the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS). From pre-selected 431 CrIS channels, 207 channels are newly selected using a one-dimensional variational (1D-Var) approach where the channel score index (CSI) is used as a figure of merit. Newly selected 207 channels consist of 85 temperature, 49 water vapor, and 73 surface channels, respectively. In addition, to examine how the channels are selected if the forecast error covariance is differently defined depending on the latitudinal regions (i.e., Northern and Southern Hemispheres, and tropics), the same selection process is carried out repeatedly using three regional forecast error covariances. From three regional channel sets, two-channel sets are made for the global data assimilation. One channel set is made with 134 channels overlapped between three regional channel sets. Another channel set consists of 277 channels that is the sum of three regional channel sets. In the global trial experiments, the global CrIS 207 channels have a significant positive forecast impact in terms of the improvement of GFS global forecasting, as compared with the forecasts with the operational 100 channels as well as the overlapped 134 and the union 277 channel sets. The improved forecast is mainly due to the additional temperature/water vapor channels of the global CrIS 207 channels that are selected optimally based on the global forecast error of operational GFS.

Download Full-text

Human Mobility Estimation Following Massive Disaster Using Filtering Approach

Journal of Disaster Research ◽

10.20965/jdr.2016.p0217 ◽

2016 ◽

Vol 11 (2) ◽

pp. 217-224 ◽

Cited By ~ 7

Author(s):

Akihito Sudo ◽

◽

Takehiro Kashiyama ◽

Takahiro Yabe ◽

Hiroshi Kanasugi ◽

...

Keyword(s):

Data Assimilation ◽

Real Time ◽

Mobile Phones ◽

Large Scale ◽

Population Distribution ◽

Human Mobility ◽

Time Estimation ◽

Observation Data ◽

Real Time Estimation ◽

Using Data

Real-time estimation of people distribution immediately after a disaster is directly related to disaster reduction and is also highly beneficial in society. Recently, traffic estimation research has been actively performed using data assimilation techniques for observation data obtained from mobile phones. However, there has been no research on data assimilation technique using real-time gridded aggregated observation data obtained from mobile phones, which are available and can be used to estimate population flow and distribution in a metropolitan area during a large-scale disaster. In this research, population distribution in an urban area during a disaster was estimated using gridded aggregated observation data obtained from mobile phones, using particle filter. The experimental results indicated that the particle filters enabled high-precision real-time estimation in the Kanto district.

Download Full-text

Effect of the Ushant SST front on regional weather forecasting in the event of anticyclonic events

10.5194/egusphere-egu2020-22412 ◽

2020 ◽

Author(s):

Francoise Orain ◽

Marie-Noelle Bouin ◽

Jean-Luc Redelsperger ◽

Valérie Garnier

Keyword(s):

Weather Forecasting ◽

Numerical Models ◽

Weather Forecast ◽

Satellite Observation ◽

Observation Data ◽

Satellite Sst ◽

Hydrostatic Model ◽

Definition Of ◽

The Impact ◽

North And South

The representation of the Ushant front in Meteo-France numerical models is not accurate. The aim of this study is to evaluate the impact of a better representation of this front derived from SST satellite observation data on the weather forecast in Brittany.&#160;The study consisted in finding and selecting cases from 2016 to 2018 where the Ushant front was present in satellite SST analysis (high spatial and temporal resolution ) with differences in weather pattern between North and South Brittany. Then compare this to the operational Arome model (Meteo France non hydrostatic model).Situations of disagreement between the model and the observations were selected. Some weather forecast simulations with Mesonh model (very close to Arome) were performed on these cases with a better definition of the Ushant front. We present some results.&#160;&#160;

Download Full-text

Big Data Assimilation: Real-time Demonstration Experiment of 30-second-update Forecasting in Tokyo in August 2020

10.5194/egusphere-egu21-6890 ◽

2021 ◽

Author(s):

Takemasa Miyoshi ◽

Takumi Honda ◽

Arata Amemiya ◽

Shigenori Otsuka ◽

Yasumitsu Maejima ◽

...

Keyword(s):

Big Data ◽

Data Assimilation ◽

Boundary Conditions ◽

Real Time ◽

Input Data ◽

Weather Prediction ◽

Computational Time ◽

Doppler Velocity ◽

Observation Data ◽

Paralympic Games

The Japan&#8217;s Big Data Assimilation (BDA) project started in October 2013 and ended its 5.5-year period in March 2019. Here, we developed a novel numerical weather prediction (NWP) system at 100-m resolution updated every 30 seconds for precise prediction of individual convective clouds. This system was designed to fully take advantage of the phased array weather radar (PAWR) which observes reflectivity and Doppler velocity at 30-second frequency for 100 elevation angles at 100-m range resolution. By the end of the 5.5-year project period, we achieved less than 30-second computational time using the Japan&#8217;s flagship K computer, whose 10-petaflops performance was ranked #1 in the TOP500 list in 2011, for past cases with all input data such as boundary conditions and observation data being ready to use. The direct follow-on project started in April 2019 under the Japan Science and Technology Agency (JST) AIP (Advanced Intelligence Project) Acceleration Research. We continued the development to achieve real-time operations of this novel 30-second-update NWP system for demonstration at the time of the Tokyo 2020 Olympic and Paralympic games. The games were postponed, but the project achieved real-time demonstration of the 30-second-update NWP system at 500-m resolution using a powerful supercomputer called Oakforest-PACS operated jointly by the Tsukuba University and the University of Tokyo. The additional developments include parameter tuning for more accurate prediction and complete workflow to prepare all input data in real time, i.e., fast data transfer from the novel dual-polarization PAWR called MP-PAWR in Saitama University, and real-time nested-domain forecasts at 18-km, 6-km, and 1.5-km to provide lateral boundary conditions for the innermost 500-m-mesh domain. A real-time test was performed during July 31 and August 7, 2020 and resulted in the actual lead time of more than 27 minutes for 30-minute prediction with very few exceptions of extended delay. Past case experiments showed that this system could capture rapid intensification and decays of convective rains that occurred in the order of less than 10 minutes, while the JMA nowcasting did not predict the rapid changes by its design. This presentation will summarize the real-time demonstration during August 25 and September 7 when Tokyo 2020 Paralympic games were supposed to take place.

Download Full-text

Influence of Aeolus data assimilation on the representation of gravity waves in ECMWF analysis fields

10.5194/egusphere-egu2020-9546 ◽

2020 ◽

Author(s):

Isabell Krisch ◽

Michael Rennie ◽

Bernd Kaifler ◽

Sonja Gisinger ◽

Oliver Reitebuch ◽

...

Keyword(s):

Data Assimilation ◽

Gravity Waves ◽

Weather Forecasting ◽

Weather Prediction ◽

European Space Agency ◽

Measurement Data ◽

Weather Forecast ◽

Medium Range Weather Forecast ◽

Space Agency ◽

Airborne Measurement

In January 2020, the European Centre for Medium-Range Weather Forecast (ECMWF) became the first numerical weather prediction (NWP) centre to assimilate wind observations from the new European Space Agency (ESA)&#8217;s Earth Explorer satellite Aeolus for operational forecasting. Aeolus was launched into space on August 22nd, 2018, carrying the world&#8217;s first spaceborne wind lidar, the Atmospheric Laser Doppler Instrument (ALADIN). ALADIN measures profiles of line-of-sight wind components from 30km altitude down to the Earth&#8217;s surface or to the level where the lidar signal is attenuated by optically thick clouds.Impact assessment studies performed at ECMWF in 2019, show improved weather forecasting skills, by assimilating Aeolus wind measurements. As a side effect, these impact experiments also reveal an influence of Aeolus data assimilation on the representation of resolved gravity waves in the ECMWF model fields. Both, orographic and non-orographic gravity waves are impacted by the Aeolus data assimilation.This impact of Aeolus data assimilation on the representation of gravity waves in ECMWF will be presented for selected case studies in the southern hemisphere. Ground-based and airborne measurement data from the SOUTHTRAC campaign will be used for validation where available.

Download Full-text

Big Data Assimilation: Real-time Workflow for 30-second-update Forecasting and Perspectives toward DA-AI Integration

10.5194/egusphere-egu2020-2483 ◽

2020 ◽

Author(s):

Takemasa Miyoshi ◽

Takmi Honda ◽

Shigenori Otsuka ◽

Arata Amemiya ◽

Yasumitsu Maejima ◽

...

Keyword(s):

Big Data ◽

Deep Learning ◽

Data Assimilation ◽

Real Time ◽

High Performance ◽

Short Term Memory ◽

Weather Forecasting ◽

Learning Algorithm ◽

Weather Prediction ◽

Scale Model

The Japan&#8217;s Big Data Assimilation (BDA) project started in October 2013 and ended its 5.5-year period in March 2019. The direct follow-on project was accepted and started in April 2019 under the Japan Science and Technology Agency (JST) AIP (Advanced Intelligence Project) Acceleration Research, with emphases on the connection with AI technologies, in particular, an integration of DA and AI with high-performance computation (HPC). The BDA project aimed to fully take advantage of &#8220;big data&#8221; from advanced sensors such as the phased array weather radar (PAWR) and Himawari-8 geostationary satellite, which provide two orders of magnitude more data than the previous sensors. We have achieved successful case studies with newly-developed 30-second-update, 100-m-mesh numerical weather prediction (NWP) system based on the RIKEN&#8217;s SCALE model and local ensemble transform Kalman filter (LETKF) to assimilate PAWR in Osaka and Kobe. We have been actively developing the workflow for real-time weather forecasting in Tokyo in summer 2020. In addition, we developed two precipitation nowcasting systems with the every-30-second PAWR data: one with an optical-flow-based system, the other with a deep-learning-based system. We chose the convolutional Long Short Term Memory (Conv-LSTM) as a deep learning algorithm, and found it effective for precipitation nowcasting. The use of Conv-LSTM would lead to an integration of DA and AI with HPC. This presentation will include an overview of the BDA project toward a DA-AI-HPC integration under the new AIP Acceleration Research scheme, and recent progress of the project.

Download Full-text