scholarly journals Bilateral Sensitivity Analysis for Understandable Neural Networks and its application to Reservoir Engineering

2020 ◽  
Author(s):  
Jian Wang ◽  
Huaqing Zhang ◽  
Kai Zhang ◽  
Nikhil Ranjan Pal
Keyword(s):  
Neural Networks ◽  
Sensitivity Analysis ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Water Injection ◽  
Superior Performance ◽  
Reservoir Engineering ◽  
Model Independent ◽  
The Relationship ◽  
Subsequent Layer

<div>In this paper, a model-independent sensitivity analysis</div><div>for (deep) neural network, Bilateral Sensitivity Analysis (BiSA), is proposed to measure the relationship between neurons and layers. Both the BiSA between pair of layers and the BiSA between any pair neurons in different layers are defined for (deep) neural networks. This sensitivity can measure the influence or contribution from any layer to another layer behind this layer in the (deep) neural networks. It provides a helpful tool to interpret the learned model. The BiSA can also measure the influence or contribution from any neuron to another neuron in a subsequent layer and is critical to analyze the relationship between neurons in different layers. Then the BiSA from any input to any output of a network is easily defined to assess the connections between the inputs and outputs. The proposed BiSA of (deep) neural networks is then applied to characterize the well connectivity in reservoir engineering. Given a network trained by Water Injection Rates (WIRs) and Liquid Production Rates (LPRs) data, the well connectivity can be efficiently described through BiSA. The empirical results verify the effectiveness of</div><div>the proposed method. The comparisons with the exiting methods demonstrate the robustness and the superior performance of the proposed method.</div>

scholarly journals Bilateral Sensitivity Analysis for Understandable Neural Networks and its application to Reservoir Engineering

2020 ◽  
Author(s):  
Jian Wang ◽  
Huaqing Zhang ◽  
Kai Zhang ◽  
Nikhil Ranjan Pal
Keyword(s):  
Neural Networks ◽  
Sensitivity Analysis ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Water Injection ◽  
Superior Performance ◽  
Reservoir Engineering ◽  
Model Independent ◽  
The Relationship ◽  
Subsequent Layer

<div>In this paper, a model-independent sensitivity analysis</div><div>for (deep) neural network, Bilateral Sensitivity Analysis (BiSA), is proposed to measure the relationship between neurons and layers. Both the BiSA between pair of layers and the BiSA between any pair neurons in different layers are defined for (deep) neural networks. This sensitivity can measure the influence or contribution from any layer to another layer behind this layer in the (deep) neural networks. It provides a helpful tool to interpret the learned model. The BiSA can also measure the influence or contribution from any neuron to another neuron in a subsequent layer and is critical to analyze the relationship between neurons in different layers. Then the BiSA from any input to any output of a network is easily defined to assess the connections between the inputs and outputs. The proposed BiSA of (deep) neural networks is then applied to characterize the well connectivity in reservoir engineering. Given a network trained by Water Injection Rates (WIRs) and Liquid Production Rates (LPRs) data, the well connectivity can be efficiently described through BiSA. The empirical results verify the effectiveness of</div><div>the proposed method. The comparisons with the exiting methods demonstrate the robustness and the superior performance of the proposed method.</div>

scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

scholarly journals An efficient pruning scheme of deep neural networks for Internet of Things applications

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chen Qi ◽  
Shibo Shen ◽  
Rongpeng Li ◽  
Zhifeng Zhao ◽  
Qing Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internet Of Things ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Compact Structure ◽  
Resource Limited ◽  
Benchmark Datasets ◽  
Iot Devices

AbstractNowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.

Inclusion of Multiple Cycling of the Potential into Deep Neural Network Classification of Voltammetric Reaction Mechanisms

2021 ◽  
Author(s):  
Luke Gundry ◽  
Gareth Kennedy ◽  
Alan Bond ◽  
Jie Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reaction Mechanisms ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Neural Network Classification ◽  
Initial Cycle

The use of Deep Neural Networks (DNNs) for the classification of electrochemical mechanisms based on training with simulations of the initial cycle of potential have been reported. In this paper,...

Automated vessel segmentation in lung CT and CTA images via deep neural networks

2021 ◽  
pp. 1-15
Author(s):  
Wenjun Tan ◽  
Luyu Zhou ◽  
Xiaoshuo Li ◽  
Xiaoyu Yang ◽  
Yufei Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Computed Tomography ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Vessel Segmentation ◽  
Multi Scale ◽  
Vascular Segmentation ◽  
Study Results ◽  
Lung Ct

BACKGROUND: The distribution of pulmonary vessels in computed tomography (CT) and computed tomography angiography (CTA) images of lung is important for diagnosing disease, formulating surgical plans and pulmonary research. PURPOSE: Based on the pulmonary vascular segmentation task of International Symposium on Image Computing and Digital Medicine 2020 challenge, this paper reviews 12 different pulmonary vascular segmentation algorithms of lung CT and CTA images and then objectively evaluates and compares their performances. METHODS: First, we present the annotated reference dataset of lung CT and CTA images. A subset of the dataset consisting 7,307 slices for training and 3,888 slices for testing was made available for participants. Second, by analyzing the performance comparison of different convolutional neural networks from 12 different institutions for pulmonary vascular segmentation, the reasons for some defects and improvements are summarized. The models are mainly based on U-Net, Attention, GAN, and multi-scale fusion network. The performance is measured in terms of Dice coefficient, over segmentation ratio and under segmentation rate. Finally, we discuss several proposed methods to improve the pulmonary vessel segmentation results using deep neural networks. RESULTS: By comparing with the annotated ground truth from both lung CT and CTA images, most of 12 deep neural network algorithms do an admirable job in pulmonary vascular extraction and segmentation with the dice coefficients ranging from 0.70 to 0.85. The dice coefficients for the top three algorithms are about 0.80. CONCLUSIONS: Study results show that integrating methods that consider spatial information, fuse multi-scale feature map, or have an excellent post-processing to deep neural network training and optimization process are significant for further improving the accuracy of pulmonary vascular segmentation.

scholarly journals Critical Assessment of Artificial Intelligence Methods for Prediction of hERG Channel Inhibition in the ‘Big Data’ Era

2020 ◽  
Author(s):  
Vishal Babu Siramshetty ◽  
Dac-Trung Nguyen ◽  
Natalia J. Martinez ◽  
Anton Simeonov ◽  
Noel T. Southall ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Big Data ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Prediction Models ◽  
Chemical Space ◽  
Superior Performance ◽  
Gradient Boosting ◽  
Artificial Intelligence Methods

The rise of novel artificial intelligence methods necessitates a comparison of this wave of new approaches with classical machine learning for a typical drug discovery project. Inhibition of the potassium ion channel, whose alpha subunit is encoded by human Ether-à-go-go-Related Gene (hERG), leads to prolonged QT interval of the cardiac action potential and is a significant safety pharmacology target for the development of new medicines. Several computational approaches have been employed to develop prediction models for assessment of hERG liabilities of small molecules including recent work using deep learning methods. Here we perform a comprehensive comparison of prediction models based on classical (random forests and gradient boosting) and modern (deep neural networks and recurrent neural networks) artificial intelligence methods. The training set (~9000 compounds) was compiled by integrating hERG bioactivity data from ChEMBL database with experimental data generated from an in-house, high-throughput thallium flux assay. We utilized different molecular descriptors including the latent descriptors, which are real-valued continuous vectors derived from chemical autoencoders trained on a large chemical space (> 1.5 million compounds). The models were prospectively validated on ~840 in-house compounds screened in the same thallium flux assay. The deep neural networks performed significantly better than the classical methods with the latent descriptors. The recurrent neural networks that operate on SMILES provided highest model sensitivity. The best models were merged into a consensus model that offered superior performance compared to reference models from academic and commercial domains. Further, we shed light on the potential of artificial intelligence methods to exploit the chemistry big data and generate novel chemical representations useful in predictive modeling and tailoring new chemical space.<br>

scholarly journals Extensive deep neural networks for transferring small scale learning to large scale systems

2019 ◽  
Vol 10 (15) ◽  
pp. 4129-4140 ◽  
Author(s):  
Kyle Mills ◽  
Kevin Ryczko ◽  
Iryna Luchak ◽  
Adam Domurad ◽  
Chris Beeler ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Large Scale ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Small Scale ◽  
Large Scale Systems ◽  
Energy Entropy ◽  
Large Systems ◽  
Number Of Particles

We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with scaling.

scholarly journals Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality

2018 ◽  
Vol 129 (4) ◽  
pp. 649-662 ◽  
Author(s):  
Christine K. Lee ◽  
Ira Hofer ◽  
Eilon Gabel ◽  
Pierre Baldi ◽  
Maxime Cannesson
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Hospital Mortality ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Operating Characteristics ◽  
Physical Status ◽  
Physical Status Classification ◽  
Receiver Operating Characteristics Curve ◽  
Status Classification

Abstract Editor’s Perspective What We Already Know about This Topic What This Article Tells Us That Is New Background The authors tested the hypothesis that deep neural networks trained on intraoperative features can predict postoperative in-hospital mortality. Methods The data used to train and validate the algorithm consists of 59,985 patients with 87 features extracted at the end of surgery. Feed-forward networks with a logistic output were trained using stochastic gradient descent with momentum. The deep neural networks were trained on 80% of the data, with 20% reserved for testing. The authors assessed improvement of the deep neural network by adding American Society of Anesthesiologists (ASA) Physical Status Classification and robustness of the deep neural network to a reduced feature set. The networks were then compared to ASA Physical Status, logistic regression, and other published clinical scores including the Surgical Apgar, Preoperative Score to Predict Postoperative Mortality, Risk Quantification Index, and the Risk Stratification Index. Results In-hospital mortality in the training and test sets were 0.81% and 0.73%. The deep neural network with a reduced feature set and ASA Physical Status classification had the highest area under the receiver operating characteristics curve, 0.91 (95% CI, 0.88 to 0.93). The highest logistic regression area under the curve was found with a reduced feature set and ASA Physical Status (0.90, 95% CI, 0.87 to 0.93). The Risk Stratification Index had the highest area under the receiver operating characteristics curve, at 0.97 (95% CI, 0.94 to 0.99). Conclusions Deep neural networks can predict in-hospital mortality based on automatically extractable intraoperative data, but are not (yet) superior to existing methods.

scholarly journals An Intelligent Model to Predict Energy Performances of Residential Buildings Based on Deep Neural Networks

Energies ◽  
2020 ◽  
Vol 13 (3) ◽  
pp. 571 ◽  
Author(s):  
Azadeh Sadeghi ◽  
Roohollah Younes Sinaki ◽  
William A. Young ◽  
Gary R. Weckman
Keyword(s):  
Neural Networks ◽  
Sensitivity Analysis ◽  
Goodness Of Fit ◽  
Deep Neural Networks ◽  
Residential Buildings ◽  
Independent Set ◽  
Energy Performance ◽  
Processing Technique ◽  
Building Structures ◽  
Wall Area

As the level of greenhouse gas emissions increases, so does the importance of the energy performance of buildings (EPB). One of the main factors to measure EPB is a structure’s heating load (HL) and cooling load (CL). HLs and CLs depend on several variables, such as relative compactness, surface area, wall area, roof area, overall height, orientation, glazing area, and glazing area distribution. This research uses deep neural networks (DNNs) to forecast HLs and CLs for a variety of structures. The DNNs explored in this research include multi-layer perceptron (MLP) networks, and each of the models in this research was developed through extensive testing with a myriad number of layers, process elements, and other data preprocessing techniques. As a result, a DNN is shown to be an improvement for modeling HLs and CLs compared to traditional artificial neural network (ANN) models. In order to extract knowledge from a trained model, a post-processing technique, called sensitivity analysis (SA), was applied to the model that performed the best with respect to the selected goodness-of-fit metric on an independent set of testing data. There are two forms of SA—local and global methods—but both have the same purpose in terms of determining the significance of independent variables within a model. Local SA assumes inputs are independent of each other, while global SA does not. To further the contribution of the research presented within this article, the results of a global SA, called state-based sensitivity analysis (SBSA), are compared to the results obtained from a traditional local technique, called sensitivity analysis about the mean (SAAM). The results of the research demonstrate an improvement over existing conclusions found in literature, which is of particular interest to decision-makers and designers of building structures.

Sign in / Sign up

Export Citation Format

Share Document