Model Uncertainty in Deep Learning Simulation of Daily Streamflow with Monte Carlo Dropout

2021 ◽  
Author(s):  
Sadegh Sadeghi Tabas ◽  
Vidya Samadi
Keyword(s):  
Monte Carlo ◽  
Deep Learning ◽  
Model Uncertainty ◽  
Random Perturbation ◽  
Drainage System ◽  
Training Dataset ◽  
Streamflow Prediction ◽  
Spatial And Temporal Scales ◽  
Daily Streamflow ◽  
Wide Range

<p>Deep Learning (DL) is becoming an increasingly important tool to produce accurate streamflow prediction across a wide range of spatial and temporal scales. However, classical DL networks do not incorporate uncertainty information but only return a point prediction. Monte-Carlo Dropout (MC-Dropout) approach offers a mathematically grounded framework to reason about DL uncertainty which was used here as random diagonal matrices to introduce randomness to the streamflow prediction process. This study employed Recurrent Neural Networks (RNNs) to simulate daily streamflow records across a coastal plain drainage system, i.e., the Northeast Cape Fear River Basin, North Carolina, USA. We employed MC-Dropout approach with the DL algorithm to make streamflow simulation more robust to potential overfitting by introducing random perturbation during training period. Daily streamflow was calibrated during 2000-2010 and validated during 2010-2014 periods. Our results provide a unique and strong evidence that variational sampling via MC-Dropout acts as a dissimilarity detector. The MC-Dropout method successfully captured the predictive error after tuning a hyperparameter on a representative training dataset. This approach was able to mitigate the problem of representing model uncertainty in DL simulations without sacrificing computational complexity or accuracy metrics and can be used for all kind of DL-based streamflow (time-series) model training with dropout.</p>

scholarly journals Development of a generative adversarial neural network for identification of potential HIV-1 inhibitors by deep learning methods

Informatics ◽  
2020 ◽  
Vol 17 (1) ◽  
pp. 7-17
Author(s):  
G. I. Nikolaev ◽  
N. A. Shuldov ◽  
A. I. Anishenko, ◽  
A. V. Tuzikov ◽  
A. M. Andrianov
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Rational Design ◽  
Binding Free Energy ◽  
Viral Envelope ◽  
Training Dataset ◽  
Learning Methods ◽  
The Neural Network ◽  
Wide Range ◽  
Hiv 1

A generative adversarial autoencoder for the rational design of potential HIV-1 entry inhibitors able to block the region of the viral envelope protein gp120 critical for the virus binding to cellular receptor CD4 was developed using deep learning methods. The research were carried out to create the  architecture of the neural network, to form  virtual compound library of potential anti-HIV-1 agents for training the neural network, to make  molecular docking of all compounds from this library with gp120, to  calculate the values of binding free energy, to generate molecular fingerprints for chemical compounds from the training dataset. The training the neural network was implemented followed by estimation of the learning outcomes and work of the autoencoder.  The validation of the neural network on a wide range of compounds from the ZINC database was carried out. The use of the neural network in combination with virtual screening of chemical databases was shown to form a productive platform for identifying the basic structures promising for the design of novel antiviral drugs that inhibit the early stages of HIV infection.

Deep Neural Networks for Hierarchical Taxonomic Fossil Classification of Carbonate Skeletal Grains

2021 ◽  
Author(s):  
Sidhant Idgunji ◽  
Madison Ho ◽  
Jonathan L. Payne ◽  
Daniel Lehrmann ◽  
Michele Morsilli ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Learning Algorithm ◽  
Training Dataset ◽  
Single Class ◽  
Deep Learning Algorithm ◽  
Human Ability ◽  
Wide Range ◽  
Potential Applications ◽  
Animal Phyla

<p>The growing digitization of fossil images has vastly improved and broadened the potential application of big data and machine learning, particularly computer vision, in paleontology. Recent studies show that machine learning is capable of approaching human abilities of classifying images, and with the increase in computational power and visual data, it stands to reason that it can match human ability but at much greater efficiency in the near future. Here we demonstrate this potential of using deep learning to identify skeletal grains at different levels of the Linnaean taxonomic hierarchy. Our approach was two-pronged. First, we built a database of skeletal grain images spanning a wide range of animal phyla and classes and used this database to train the model. We used a Python-based method to automate image recognition and extraction from published sources. Second, we developed a deep learning algorithm that can attach multiple labels to a single image. Conventionally, deep learning is used to predict a single class from an image; here, we adopted a Branch Convolutional Neural Network (B-CNN) technique to classify multiple taxonomic levels for a single skeletal grain image. Using this method, we achieved over 90% accuracy for both the coarse, phylum-level recognition and the fine, class-level recognition across diverse skeletal grains (6 phyla and 15 classes). Furthermore, we found that image augmentation improves the overall accuracy. This tool has potential applications in geology ranging from biostratigraphy to paleo-bathymetry, paleoecology, and microfacies analysis. Further improvement of the algorithm and expansion of the training dataset will continue to narrow the efficiency gap between human expertise and machine learning.</p>

scholarly journals Pano-RSOD: A Dataset and Benchmark for Panoramic Road Scene Object Detection

Electronics ◽  
2019 ◽  
Vol 8 (3) ◽  
pp. 329 ◽  
Author(s):  
Yong Li ◽  
Guofeng Tong ◽  
Huashuai Gao ◽  
Yuebin Wang ◽  
Liqiang Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Training Data ◽  
Training Dataset ◽  
Panoramic Image ◽  
Information Object ◽  
Wide Range ◽  
Panoramic Images ◽  
Bounding Boxes ◽  
Scene Object

Panoramic images have a wide range of applications in many fields with their ability to perceive all-round information. Object detection based on panoramic images has certain advantages in terms of environment perception due to the characteristics of panoramic images, e.g., lager perspective. In recent years, deep learning methods have achieved remarkable results in image classification and object detection. Their performance depends on the large amount of training data. Therefore, a good training dataset is a prerequisite for the methods to achieve better recognition results. Then, we construct a benchmark named Pano-RSOD for panoramic road scene object detection. Pano-RSOD contains vehicles, pedestrians, traffic signs and guiding arrows. The objects of Pano-RSOD are labelled by bounding boxes in the images. Different from traditional object detection datasets, Pano-RSOD contains more objects in a panoramic image, and the high-resolution images have 360-degree environmental perception, more annotations, more small objects and diverse road scenes. The state-of-the-art deep learning algorithms are trained on Pano-RSOD for object detection, which demonstrates that Pano-RSOD is a useful benchmark, and it provides a better panoramic image training dataset for object detection tasks, especially for small and deformed objects.

Addressing model uncertainty in probabilistic forecasting using Monte Carlo dropout

2020 ◽  
Vol 24 ◽  
pp. 185-205
Author(s):  
Cristián Serpell ◽  
Ignacio A. Araya ◽  
Carlos Valle ◽  
Héctor Allende
Keyword(s):  
Monte Carlo ◽  
Deep Learning ◽  
Model Uncertainty ◽  
Variational Approach ◽  
Variance Estimation ◽  
Learning Models ◽  
Probabilistic Forecasting ◽  
Deep Model ◽  
Uncertain Future ◽  
Mean Variance

In recent years, deep learning models have been developed to address probabilistic forecasting tasks, assuming an implicit stochastic process that relates past observed values to uncertain future values. These models are capable of capturing the inherent uncertainty of the underlying process, but they ignore the model uncertainty that comes from the fact of not having infinite data. This work proposes addressing the model uncertainty problem using Monte Carlo dropout, a variational approach that assigns distributions to the weights of a neural network instead of simply using fixed values. This allows to easily adapt common deep learning models currently in use to produce better probabilistic forecasting estimates, in terms of their consideration of uncertainty. The proposal is validated for prediction intervals estimation on seven energy time series, using a popular probabilistic model called Mean Variance Estimation (MVE), as the deep model adapted using the technique.

scholarly journals Combining Crowdsourcing and Deep Learning to Explore the Mesoscale Organization of Shallow Convection

2020 ◽  
Vol 101 (11) ◽  
pp. E1980-E1995 ◽  
Author(s):  
Stephan Rasp ◽  
Hauke Schulz ◽  
Sandrine Bony ◽  
Bjorn Stevens
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Climate Models ◽  
Training Dataset ◽  
Trade Wind ◽  
Radiation Balance ◽  
Shallow Convection ◽  
Wide Range ◽  
Shallow Clouds ◽  
New Phenomena

AbstractHumans excel at detecting interesting patterns in images, for example, those taken from satellites. This kind of anecdotal evidence can lead to the discovery of new phenomena. However, it is often difficult to gather enough data of subjective features for significant analysis. This paper presents an example of how two tools that have recently become accessible to a wide range of researchers, crowdsourcing and deep learning, can be combined to explore satellite imagery at scale. In particular, the focus is on the organization of shallow cumulus convection in the trade wind regions. Shallow clouds play a large role in the Earth’s radiation balance yet are poorly represented in climate models. For this project four subjective patterns of organization were defined: Sugar, Flower, Fish, and Gravel. On cloud-labeling days at two institutes, 67 scientists screened 10,000 satellite images on a crowdsourcing platform and classified almost 50,000 mesoscale cloud clusters. This dataset is then used as a training dataset for deep learning algorithms that make it possible to automate the pattern detection and create global climatologies of the four patterns. Analysis of the geographical distribution and large-scale environmental conditions indicates that the four patterns have some overlap with established modes of organization, such as open and closed cellular convection, but also differ in important ways. The results and dataset from this project suggest promising research questions. Further, this study illustrates that crowdsourcing and deep learning complement each other well for the exploration of image datasets.

scholarly journals Uncertainty Estimation with Deep Learning for Rainfall–Runoff Modelling

2021 ◽  
Author(s):  
Daniel Klotz ◽  
Frederik Kratzert ◽  
Martin Gauch ◽  
Alden Keefe Sampson ◽  
Johannes Brandstetter ◽  
...  
Keyword(s):  
Deep Learning ◽  
Hydrological Model ◽  
Model Development ◽  
Uncertainty Estimation ◽  
Rainfall Runoff ◽  
Spatial And Temporal Scales ◽  
Mixture Density ◽  
Qualitative Understanding ◽  
Wide Range ◽  
Post Hoc

Abstract. Deep Learning is becoming an increasingly important way to produce accurate hydrological predictions across a wide range of spatial and temporal scales. Uncertainty estimations are critical for actionable hydrological forecasting, and while standardized community benchmarks are becoming an increasingly important part of hydrological model development and research, similar tools for benchmarking uncertainty estimation are lacking. This contributions demonstrates that accurate uncertainty predictions can be obtained with Deep Learning. We establish an uncertainty estimation benchmarking procedure and present four Deep Learning baselines. Three baselines are based on Mixture Density Networks and one is based on Monte Carlo dropout. The results indicate that these approaches constitute strong baselines, especially the former ones. Additionaly, we provide a post-hoc model analysis to put forward some qualitative understanding of the resulting models. This analysis extends the notion of performance and show that learn nuanced behaviors in different situations.

scholarly journals Automated Graduation using Bayesian Trans-dimensional Models

2011 ◽  
Vol 5 (2) ◽  
pp. 231-251 ◽  
Author(s):  
R.J. Verrall ◽  
S. Haberman
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Bayesian Approach ◽  
Model Uncertainty ◽  
Weighted Average ◽  
New Method ◽  
Reversible Jump ◽  
Standard Methods ◽  
Wide Range

AbstractThis paper presents a new method of graduation which uses parametric formulae together with Bayesian reversible jump Markov chain Monte Carlo methods. The aim is to provide a method which can be applied to a wide range of data, and which does not require a lot of adjustment or modification. The method also does not require one particular parametric formula to be selected: instead, the graduated values are a weighted average of the values from a range of formulae. In this way, the new method can be seen as an automatic graduation method which we believe can be applied in many cases without any adjustments and provide satisfactory graduated values. An advantage of a Bayesian approach is that it allows for model uncertainty unlike standard methods of graduation.

scholarly journals Cell segmentation using deep learning: comparing label and label-free approaches using hyper-labeled image stacks

2020 ◽  
Author(s):  
William D. Cameron ◽  
Alex M. Bennett ◽  
Cindy V. Bui ◽  
Huntley H. Chang ◽  
Jonathan V. Rocheleau
Keyword(s):  
Deep Learning ◽  
Cell Segmentation ◽  
Training Dataset ◽  
Label Free ◽  
Final Model ◽  
Training Models ◽  
Fluorescent Markers ◽  
Wide Range ◽  
And Training ◽  
Labeling Strategies

AbstractDeep learning provides an opportunity to automatically segment and extract cellular features from high-throughput microscopy images. Many labeling strategies have been developed for this purpose, ranging from the use of fluorescent markers to label-free approaches. However, differences in the channels available to each respective training dataset make it difficult to directly compare the effectiveness of these strategies across studies. Here we explore training models using subimage stacks composed of channels sampled from larger, ‘hyper-labeled’, image stacks. This allows us to directly compare a variety of labeling strategies and training approaches on identical cells. This approach revealed that fluorescence-based strategies generally provide higher segmentation accuracies but were less accurate than label-free models when labeling was inconsistent. The relative strengths of label and label-free techniques could be combined through the use of merging fluorescence channels and out-of-focus brightfield images. Beyond comparing labeling strategies, using subimage stacks for training was also found to provide a method of simulating a wide range of labeling conditions, increasing the ability of the final model to accommodate a greater range of experimental setups.

Monte Carlo calculation of the energy parameters and spatial distribution of the cathodic arc ions while passing through the macro-particles filters

2018 ◽  
Vol 1 (1) ◽  
pp. 30-34 ◽  
Author(s):  
Alexey Chernogor ◽  
Igor Blinkov ◽  
Alexey Volkhonskiy
Keyword(s):  
Magnetic Field ◽  
Mass Transfer ◽  
Monte Carlo ◽  
Monte Carlo Calculation ◽  
Inhomogeneous Magnetic Field ◽  
Flow Energy ◽  
Wide Range ◽  
Field Concentrator ◽  
Cathodic Arc ◽  
The Magnetic Field

The flow, energy distribution and concentrations profiles of Ti ions in cathodic arc are studied by test particle Monte Carlo simulations with considering the mass transfer through the macro-particles filters with inhomogeneous magnetic field. The loss of ions due to their deposition on filter walls was calculated as a function of electric current and number of turns in the coil. The magnetic field concentrator that arises in the bending region of the filters leads to increase the loss of the ions component of cathodic arc. The ions loss up to 80 % of their energy resulted by the paired elastic collisions which correspond to the experimental results. The ion fluxes arriving at the surface of the substrates during planetary rotating of them opposite the evaporators mounted to each other at an angle of 120° characterized by the wide range of mutual overlapping.

scholarly journals Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots

GigaScience ◽  
2021 ◽  
Vol 10 (5) ◽  
Author(s):  
Teng Miao ◽  
Weiliang Wen ◽  
Yinglun Li ◽  
Sheng Wu ◽  
Chao Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Training Dataset ◽  
Growth Stages ◽  
3D Point Cloud ◽  
Plant Structure ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Different Growth Stages

Abstract Background The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking. Results We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation. Conclusion Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.

Sign in / Sign up

Export Citation Format

Share Document