scholarly journals A Bayesian graph convolutional network for reliable prediction of molecular properties with uncertainty quantification

2019 ◽  
Vol 10 (36) ◽  
pp. 8438-8446 ◽  
Author(s):  
Seongok Ryu ◽  
Yongchan Kwon ◽  
Woo Youn Kim
Keyword(s):  
Neural Networks ◽  
Bayesian Inference ◽  
Uncertainty Analysis ◽  
Uncertainty Quantification ◽  
Deep Neural Networks ◽  
Molecular Properties ◽  
Reliable Prediction ◽  
Convolutional Network ◽  
Chemical Fields

Deep neural networks have been increasingly used in various chemical fields. Here, we show that Bayesian inference enables more reliable prediction with quantitative uncertainty analysis.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals Mitigation of Effects of Occlusion on Object Recognition with Deep Neural Networks through Low-Level Image Completion

2016 ◽  
Vol 2016 ◽  
pp. 1-15 ◽  
Author(s):  
Benjamin Chandler ◽  
Ennio Mingolla
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Poor Performance ◽  
Classification Algorithms ◽  
Image Completion ◽  
Convolutional Network ◽  
Occluded Objects

Heavily occluded objects are more difficult for classification algorithms to identify correctly than unoccluded objects. This effect is rare and thus hard to measure with datasets like ImageNet and PASCAL VOC, however, owing to biases in human-generated image pose selection. We introduce a dataset that emphasizes occlusion and additions to a standard convolutional neural network aimed at increasing invariance to occlusion. An unmodified convolutional neural network trained and tested on the new dataset rapidly degrades to chance-level accuracy as occlusion increases. Training with occluded data slows this decline but still yields poor performance with high occlusion. Integrating novel preprocessing stages to segment the input and inpaint occlusions is an effective mitigation. A convolutional network so modified is nearly as effective with more than 81% of pixels occluded as it is with no occlusion. Such a network is also more accurate on unoccluded images than an otherwise identical network that has been trained with only unoccluded images. These results depend on successful segmentation. The occlusions in our dataset are deliberately easy to segment from the figure and background. Achieving similar results on a more challenging dataset would require finding a method to split figure, background, and occluding pixels in the input.

Reduced-Order Flood Modeling Using Uncertainty-Aware Deep Neural Networks

2020 ◽  
Author(s):  
Pierre Jacquier ◽  
Azzedine Abdedou ◽  
Azzeddine Soulaïmani
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Uncertainty Quantification ◽  
Surrogate Model ◽  
Deep Neural Networks ◽  
Reduced Order Modeling ◽  
Flood Modeling ◽  
Reduced Order ◽  
Highly Nonlinear ◽  
The Impact

<p><strong>Key Words</strong>: Uncertainty Quantification, Deep Learning, Space-Time POD, Flood Modeling</p><p><br>While impressive results have been achieved in the well-known fields where Deep Learning allowed for breakthroughs such as computer vision, language modeling, or content generation [1], its impact on different, older fields is still vastly unexplored. In computational fluid dynamics and especially in Flood Modeling, many phenomena are very high-dimensional, and predictions require the use of finite element or volume methods, which can be, while very robust and tested, computational-heavy and may not prove useful in the context of real-time predictions. This led to various attempts at developing Reduced-Order Modeling techniques, both intrusive and non-intrusive. One late relevant addition was a combination of Proper Orthogonal Decomposition with Deep Neural Networks (POD-NN) [2]. Yet, to our knowledge, in this example and more generally in the field, little work has been conducted on quantifying uncertainties through the surrogate model.<br>In this work, we aim at comparing different novel methods addressing uncertainty quantification in reduced-order models, pushing forward the POD-NN concept with ensembles, latent-variable models, as well as encoder-decoder models. These are tested on benchmark problems, and then applied to a real-life application: flooding predictions in the Mille-Iles river in Laval, QC, Canada.<br>For the flood modeling application, our setup involves a set of input parameters resulting from onsite measures. High-fidelity solutions are then generated using our own finite-volume code CuteFlow, which is solving the highly nonlinear Shallow Water Equations. The goal is then to build a non-intrusive surrogate model, that’s able to <em>know what it know</em>s, and more importantly, <em>know when it doesn’t</em>, which is still an open research area as far as neural networks are concerned [3].</p><p><br><strong>REFERENCES</strong><br>[1] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning”, in Thirty-First AAAI Conference on Artificial Intelligence, 2017.<br>[2] Q. Wang, J. S. Hesthaven, and D. Ray, “Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem”, Journal of Computational Physics, vol. 384, pp. 289–307, May 2019.<br>[3] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles”, in Advances in Neural Information Processing Systems, 2017, pp. 6402–6413.</p>

scholarly journals Flexible Bayesian Inference by Weight Transfer for Robust Deep Neural Networks

2021 ◽  
Vol E104.D (11) ◽  
pp. 1981-1991
Author(s):  
Thi Thu Thao KHONG ◽  
Takashi NAKADA ◽  
Yasuhiko NAKASHIMA
Keyword(s):  
Neural Networks ◽  
Bayesian Inference ◽  
Deep Neural Networks ◽  
Weight Transfer

scholarly journals Deep Neural Networks for the Classification of Pure and Impure Strawberry Purees

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1223 ◽  
Author(s):  
Zhong Zheng ◽  
Xin Zhang ◽  
Jinxing Yu ◽  
Rui Guo ◽  
Lili Zhangzhong
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Classification Accuracy ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Convolutional Network ◽  
Different Types ◽  
Long Short Term Memory ◽  
Gated Recurrent Unit

In this paper, a comparative study of the effectiveness of deep neural networks (DNNs) in the classification of pure and impure purees is conducted. Three different types of deep neural networks (DNNs)—the Gated Recurrent Unit (GRU), the Long Short Term Memory (LSTM), and the temporal convolutional network (TCN)—are employed for the detection of adulteration of strawberry purees. The Strawberry dataset, a time series spectroscopy dataset from the UCR time series classification repository, is utilized to evaluate the performance of different DNNs. Experimental results demonstrate that the TCN is able to obtain a higher classification accuracy than the GRU and LSTM. Moreover, the TCN achieves a new state-of-the-art classification accuracy on the Strawberry dataset. These results indicates the great potential of using the TCN for the detection of adulteration of fruit purees in the future.

scholarly journals HatchEnsemble: an efficient and practical uncertainty quantification method for deep neural networks

2021 ◽  
Author(s):  
Yufeng Xia ◽  
Jun Zhang ◽  
Tingsong Jiang ◽  
Zhiqiang Gong ◽  
Wen Yao ◽  
...  
Keyword(s):  
Neural Networks ◽  
Uncertainty Quantification ◽  
Bayesian Methods ◽  
Large Scale ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Main Idea ◽  
Quantification Theory ◽  
Complete Uncertainty ◽  
Scale Models

AbstractQuantifying predictive uncertainty in deep neural networks is a challenging and yet unsolved problem. Existing quantification approaches can be categorized into two lines. Bayesian methods provide a complete uncertainty quantification theory but are often not scalable to large-scale models. Along another line, non-Bayesian methods have good scalability and can quantify uncertainty with high quality. The most remarkable idea in this line is Deep Ensemble, but it is limited in practice due to its expensive computational cost. Thus, we propose HatchEnsemble to improve the efficiency and practicality of Deep Ensemble. The main idea is to use function-preserving transformations, ensuring HatchNets to inherit the knowledge learned by a single model called SeedNet. This process is called hatching, and HatchNet can be obtained by continuously widening the SeedNet. Based on our method, two different hatches are proposed, respectively, for ensembling the same and different architecture networks. To ensure the diversity of models, we also add random noises to parameters during hatching. Experiments on both clean and corrupted datasets show that HatchEnsemble can give a competitive prediction performance and better-calibrated uncertainty quantification in a shorter time compared with baselines.

scholarly journals Global importance analysis: An interpretability method to quantify importance of genomic features in deep neural networks

2021 ◽  
Vol 17 (5) ◽  
pp. e1008925
Author(s):  
Peter K. Koo ◽  
Antonio Majdandzic ◽  
Matthew Ploenzke ◽  
Praveen Anand ◽  
Steffan B. Paul
Keyword(s):  
Neural Networks ◽  
Protein Interactions ◽  
Effect Size ◽  
Deep Neural Networks ◽  
Rna Binding ◽  
Rna Binding Proteins ◽  
Population Level ◽  
Sequence Motifs ◽  
Convolutional Network ◽  
Importance Analysis

Deep neural networks have demonstrated improved performance at predicting the sequence specificities of DNA- and RNA-binding proteins compared to previous methods that rely on k-mers and position weight matrices. To gain insights into why a DNN makes a given prediction, model interpretability methods, such as attribution methods, can be employed to identify motif-like representations along a given sequence. Because explanations are given on an individual sequence basis and can vary substantially across sequences, deducing generalizable trends across the dataset and quantifying their effect size remains a challenge. Here we introduce global importance analysis (GIA), a model interpretability method that quantifies the population-level effect size that putative patterns have on model predictions. GIA provides an avenue to quantitatively test hypotheses of putative patterns and their interactions with other patterns, as well as map out specific functions the network has learned. As a case study, we demonstrate the utility of GIA on the computational task of predicting RNA-protein interactions from sequence. We first introduce a convolutional network, we call ResidualBind, and benchmark its performance against previous methods on RNAcompete data. Using GIA, we then demonstrate that in addition to sequence motifs, ResidualBind learns a model that considers the number of motifs, their spacing, and sequence context, such as RNA secondary structure and GC-bias.

Sign in / Sign up

Export Citation Format

Share Document