Probabilistic Predictions from Deterministic Atmospheric River Forecasts with Deep Learning

2021 ◽  
Author(s):  
William E. Chapman ◽  
Luca Delle Monache ◽  
Stefano Alessandrini ◽  
Aneesh C. Subramanian ◽  
F. Martin Ralph ◽  
...  
Keyword(s):  
Deep Learning ◽  
American West ◽  
Mesoscale Model ◽  
Skill Score ◽  
Vapor Transport ◽  
Training Dataset ◽  
Lead Times ◽  
North American West ◽  
Atmospheric River ◽  
Probabilistic Forecasts

AbstractDeep Learning (DL) post-processing methods are examined to obtain reliable and accurate probabilistic forecasts from single-member numerical weather predictions of integrated vapor transport (IVT). Using a 34-year reforecast, based on the Center for Western Weather and Water Extremes West-WRF mesoscale model of North American West Coast IVT, the dynamically/statistically derived 0-120 hour probabilistic forecasts for IVT under atmospheric river (AR) conditions are tested. These predictions are compared to the Global Ensemble Forecast System (GEFS) dynamic model and the GEFS calibrated with a neural network. Additionally, the DL methods are tested against an established, but more rigid, statistical-dynamical ensemble method (the Analog Ensemble). The findings show, using continuous ranked probability skill score and Brier skill score as verification metrics, that the DL methods compete with or outperform the calibrated GEFS system at lead times from 0-48 hours and again from 72-120 hours for AR vapor transport events. Additionally, the DL methods generate reliable and skillful probabilistic forecasts. The implications of varying the length of the training dataset are examined and the results show that the DL methods learn relatively quickly and ~10 years of hindcast data are required to compete with the GEFS ensemble.

Using deep learning to nowcast the spatial coverage of convection from Himawari-8 satellite data

2021 ◽  
Author(s):  
Ryan Lagerquist ◽  
Jebb Q. Stewart ◽  
Imme Ebert-Uphoff ◽  
Christina Kumler
Keyword(s):  
Deep Learning ◽  
Satellite Data ◽  
Geographic Location ◽  
Ground Truth ◽  
Atmospheric Science ◽  
Skill Score ◽  
Time Of Day ◽  
Lead Times ◽  
Weather Radars ◽  
Spatial Coverage

AbstractPredicting the timing and location of thunderstorms (“convection”) allows for preventive actions that can save both lives and property. We have applied U-nets, a deep-learning-based type of neural network, to forecast convection on a grid at lead times up to 120 minutes. The goal is to make skillful forecasts with only present and past satellite data as predictors. Specifically, predictors are multispectral brightness-temperature images from the Himawari-8 satellite, while targets (ground truth) are provided by weather radars in Taiwan. U-nets are becoming popular in atmospheric science due to their advantages for gridded prediction. Furthermore, we use three novel approaches to advance U-nets in atmospheric science. First, we compare three architectures – vanilla, temporal, and U-net++ – and find that vanilla U-nets are best for this task. Second, we train U-nets with the fractions skill score, which is spatially aware, as the loss function. Third, because we do not have adequate ground truth over the full Himawari-8 domain, we train the U-nets with small radar-centered patches, then apply trained U-nets to the full domain. Also, we find that the best predictions are given by U-nets trained with satellite data from multiple lag times, not only the present. We evaluate U-nets in detail – by time of day, month, and geographic location – and compare to persistence models. The U-nets outperform persistence at lead times ≥ 60 minutes, and at all lead times the U-nets provide a more realistic climatology than persistence. Our code is available publicly.

scholarly journals Rapid Cyclogenesis from a Mesoscale Frontal Wave on an Atmospheric River: Impacts on Forecast Skill and Predictability during Atmospheric River Landfall

2019 ◽  
Vol 20 (9) ◽  
pp. 1779-1794 ◽  
Author(s):  
Andrew C. Martin ◽  
F. Martin Ralph ◽  
Anna Wilson ◽  
Laurel DeHaan ◽  
Brian Kawzenuk
Keyword(s):  
Water Vapor ◽  
Lead Time ◽  
Forecast Skill ◽  
Vapor Transport ◽  
Lead Times ◽  
Forecast Uncertainty ◽  
Water Vapor Flux ◽  
Vapor Flux ◽  
Frontal Wave ◽  
Atmospheric River

Abstract Mesoscale frontal waves have the potential to modify the hydrometeorological impacts of atmospheric rivers (ARs). The small scale and rapid growth of these waves pose significant forecast challenges. We examined a frontal wave that developed a secondary cyclone during the landfall of an extreme AR in Northern California. We document rapid changes in significant storm features including integrated vapor transport and precipitation and connect these to high forecast uncertainty at 1–4-days’ lead time. We also analyze the skill of the Global Ensemble Forecast System in predicting secondary cyclogenesis and relate secondary cyclogenesis prediction skill to forecasts of AR intensity, AR duration, and upslope water vapor flux in the orographic controlling layer. Leveraging a measure of reference accuracy designed for cyclogenesis, we found forecasts were only able to skillfully predict secondary cyclogenesis for lead times less than 36 h. Forecast skill in predicting the large-scale pressure pattern and integrated vapor transport was lost by 96-h lead time. For lead times longer than 36 h, the failure to predict secondary cyclogenesis led to significant uncertainty in forecast AR intensity and to long bias in AR forecast duration. Failure to forecast a warm front associated with the secondary cyclone at lead times less than 36 h caused large overprediction of upslope water vapor flux, an important indicator of orographic precipitation forcing. This study highlights the need to identify offshore mesoscale frontal waves in real time and to characterize the forecast uncertainty inherent in these events when creating hydrometeorological forecasts.

The Naval Research Laboratory’s Coupled Ocean/Atmosphere Mesoscale Prediction System – Tropical Cyclone ensemble (COAMPS-TC ensemble)

2021 ◽  
Author(s):  
William A. Komaromi ◽  
Patrick A. Reinecke ◽  
James D. Doyle ◽  
Jonathan R. Moskaitis
Keyword(s):  
Tropical Cyclone ◽  
Boundary Conditions ◽  
Skill Score ◽  
Horizontal Resolution ◽  
Naval Research ◽  
Lead Times ◽  
Prediction System ◽  
Intensity Forecast ◽  
Probabilistic Forecasts ◽  
Ocean Atmosphere

AbstractThe 11-member Coupled Ocean/Atmosphere Mesoscale Prediction System for Tropical Cyclones (COAMPS-TC) ensemble has been developed by the Naval Research Laboratory (NRL) to produce probabilistic forecasts of tropical cyclone (TC) track, intensity and structure. All members run with a storm-following inner grid at convection-permitting 4-km horizontal resolution. The COAMPS-TC ensemble is constructed via a combination of perturbations to initial and boundary conditions, the initial vortex, and model physics to account for a variety of different sources of uncertainty that affect track and intensity forecasts. Unlike global model ensembles, which do a reasonable job capturing track uncertainty but not intensity, mesoscale ensembles such as the COAMPS-TC ensemble are necessary to provide a realistic intensity forecast spectrum.The initial and boundary condition perturbations are responsible for generating the majority of track spread at all lead times, as well as the intensity spread from 36-120 h. The vortex and physics perturbations are necessary to produce meaningful spread in the intensity prediction from 0-36 h. In a large sample of forecasts from 2014-2017, the ensemble-mean track and intensity forecast is superior to the unperturbed control forecast at all lead times, demonstrating a clear advantage to running an ensemble versus a deterministic forecast. The spread-skill relationship of the ensemble is also examined, and is found to be very well calibrated for track, but is under-dispersive for intensity. Using a mixture of lateral boundary conditions derived from different global models is found to improve upon the spread-skill score for intensity, but it is hypothesized that additional physics perturbations will be necessary to achieve realistic ensemble spread.

scholarly journals A calibrated and consistent combination of probabilistic forecasts for the exceedance of several precipitation thresholds using neural networks.

2021 ◽  
Author(s):  
Peter Schaumann ◽  
Reinhold Hess ◽  
Martin Rempel ◽  
Ulrich Blahak ◽  
Volker Schmidt
Keyword(s):  
Neural Networks ◽  
Processing System ◽  
Skill Score ◽  
Horizontal Resolution ◽  
Grid Size ◽  
Ensemble Prediction ◽  
Lead Times ◽  
Ensemble Prediction System ◽  
Probabilistic Forecasts ◽  
Precipitation Thresholds

<p>In this talk we present a new statistical method for the seamless combination of two different ensemble precipitation forecasts (Nowcasting and NWP) using neural networks (NNs), see [1]. The method generates probabilistic forecasts for the exceedance of a set of predetermined thresholds (from 0.1mm up to 5mm). The aim of the combination model is to produce seamless and calibrated forecasts which outperform both input forecasts for all lead times and which are consistent regarding the considered thresholds. First, the hyper-parameters of the NNs are chosen according to a certain hyper-parameter optimization algorithm (not to be confused with the training of the NNs itself) on a 3-month dataset (dataset A). Then, the resulting NNs are tested via a rolling origin validation scheme on two 3-month datasets (datasets B & C) with different input forecasts each. Datasets A & B contain forecasts of DWD's RadVOR, a radar-based nowcasting system, and Ensemble-MOS, a post-processing system of NWP ensembles made by COSMO-DE-EPS, with a horizontal resolution of 20km, which is a predecessor of ICON-D2-EPS. Ensemble-MOS forecasts were provided for up to +6h, while RadVOR forecasts were available up to +2h. For dataset C, forecasts with a grid size of 2.2km are used from STEPS-DWD, a new implementation of the Short-term Ensemble Prediction System (STEPS) by  DWD, and ICON-D2-EPS as a NWP ensemble system. Forecasts were made up to +6h. In both validation datasets (B & C), the forecasts show the well-known behavior that the nowcasting systems RadVOR & STEPS are superior for short lead times, while NWP forecasts (Ensemble-MOS & ICON-D2-EPS) outperform these systems for later lead times. Based on the comparison of several validation scores (bias, Brier skill score, reliability and reliability diagram) we can show that the combination is indeed calibrated, consistent and outperforms both input forecasts for all lead times. It should be noted that the combination works on dataset C, although the hyper-parameters were chosen based on dataset A, which contains different forecasts for a different grid size.<br><br>[1] P. Schaumann, R. Hess, M. Rempel, U. Blahak and V. Schmidt, A calibrated and consistent combination of probabilistic forecasts for the exceedance of several precipitation thresholds using neural networks. Weather and Forecasting (in print)</p>

The Blue Flower and the Account Book

2016 ◽  
Vol 85 (1) ◽  
pp. 1-22
Author(s):  
Anne F. Hyde
Keyword(s):  
Nineteenth Century ◽  
North American ◽  
American West ◽  
Fur Trade ◽  
Blue Flower ◽  
Racial Category ◽  
North American West ◽  
Mixed Blood ◽  
Anglo American

This essay, a revised version of the August 2015 talk, examines the story of two mixed-blood women, indigenous and Anglo American, who lived in the fur trade North American West. The essay examines a racial category, mixed blood or “half-breed” and considers the challenges for people who lived in and used that category in the nineteenth century. The essay illuminates the challenges of using different kinds of personal records to understand how these nineteenth-century women might have thought about identity, a word they never would have used.

scholarly journals Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots

GigaScience ◽  
2021 ◽  
Vol 10 (5) ◽  
Author(s):  
Teng Miao ◽  
Weiliang Wen ◽  
Yinglun Li ◽  
Sheng Wu ◽  
Chao Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Training Dataset ◽  
Growth Stages ◽  
3D Point Cloud ◽  
Plant Structure ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Different Growth Stages

Abstract Background The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking. Results We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation. Conclusion Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.

scholarly journals Study on Radar Echo-Filling in an Occlusion Area by a Deep Learning Algorithm

2021 ◽  
Vol 13 (9) ◽  
pp. 1779
Author(s):  
Xiaoyan Yin ◽  
Zhiqun Hu ◽  
Jiafeng Zheng ◽  
Boyong Li ◽  
Yuanyuan Zuo
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
Learning Algorithm ◽  
Weather Radar ◽  
Loss Functions ◽  
Training Dataset ◽  
Echo Intensity ◽  
Common Mean ◽  
Deep Learning Algorithm ◽  
Radar Beam

Radar beam blockage is an important error source that affects the quality of weather radar data. An echo-filling network (EFnet) is proposed based on a deep learning algorithm to correct the echo intensity under the occlusion area in the Nanjing S-band new-generation weather radar (CINRAD/SA). The training dataset is constructed by the labels, which are the echo intensity at the 0.5° elevation in the unblocked area, and by the input features, which are the intensity in the cube including multiple elevations and gates corresponding to the location of bottom labels. Two loss functions are applied to compile the network: one is the common mean square error (MSE), and the other is a self-defined loss function that increases the weight of strong echoes. Considering that the radar beam broadens with distance and height, the 0.5° elevation scan is divided into six range bands every 25 km to train different models. The models are evaluated by three indicators: explained variance (EVar), mean absolute error (MAE), and correlation coefficient (CC). Two cases are demonstrated to compare the effect of the echo-filling model by different loss functions. The results suggest that EFnet can effectively correct the echo reflectivity and improve the data quality in the occlusion area, and there are better results for strong echoes when the self-defined loss function is used.

scholarly journals Evaluation of the feasibility of explainable computer-aided detection of cardiomegaly on chest radiographs using deep learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Mu Sook Lee ◽  
Yong Soo Kim ◽  
Minki Kim ◽  
Muhammad Usman ◽  
Shi Sub Byon ◽  
...  
Keyword(s):  
Deep Learning ◽  
Diagnostic Performance ◽  
Absolute Error ◽  
Training Dataset ◽  
Computer Aided Detection ◽  
Test Dataset ◽  
Cardiothoracic Ratio ◽  
Computer Aided ◽  
Chest X Ray ◽  
Public Datasets

AbstractWe examined the feasibility of explainable computer-aided detection of cardiomegaly in routine clinical practice using segmentation-based methods. Overall, 793 retrospectively acquired posterior–anterior (PA) chest X-ray images (CXRs) of 793 patients were used to train deep learning (DL) models for lung and heart segmentation. The training dataset included PA CXRs from two public datasets and in-house PA CXRs. Two fully automated segmentation-based methods using state-of-the-art DL models for lung and heart segmentation were developed. The diagnostic performance was assessed and the reliability of the automatic cardiothoracic ratio (CTR) calculation was determined using the mean absolute error and paired t-test. The effects of thoracic pathological conditions on performance were assessed using subgroup analysis. One thousand PA CXRs of 1000 patients (480 men, 520 women; mean age 63 ± 23 years) were included. The CTR values derived from the DL models and diagnostic performance exhibited excellent agreement with reference standards for the whole test dataset. Performance of segmentation-based methods differed based on thoracic conditions. When tested using CXRs with lesions obscuring heart borders, the performance was lower than that for other thoracic pathological findings. Thus, segmentation-based methods using DL could detect cardiomegaly; however, the feasibility of computer-aided detection of cardiomegaly without human intervention was limited.

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

Sign in / Sign up

Export Citation Format

Share Document