Uncertainty quantification in fault detection using convolutional neural networks

Geophysics ◽  
2021 ◽  
pp. 1-45
Author(s):  
Runhai Feng ◽  
Dario Grana ◽  
Niels Balling
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Reservoir Characterization ◽  
Epistemic Uncertainty ◽  
Confidence Regions ◽  
Training Data ◽  
Model Parameters ◽  
Sigmoid Function ◽  
Learning Tools

Segmentation of faults based on seismic images is an important step in reservoir characterization. With the recent developments of deep-learning methods and the availability of massive computing power, automatic interpretation of seismic faults has become possible. The likelihood of occurrence for a fault can be quantified using a sigmoid function. Our goal is to quantify the fault model uncertainty that is generally not captured by deep-learning tools. We propose to use the dropout approach, a regularization technique to prevent overfitting and co-adaptation in hidden units, to approximate the Bayesian inference and estimate the principled uncertainty over functions. Particularly, the variance of the learned model has been decomposed into aleatoric and epistemic parts. The proposed method is applied to a real dataset from the Netherlands F3 block with two different dropout ratios in convolutional neural networks. The aleatoric uncertainty is irreducible since it relates to the stochastic dependency within the input observations. As the number of Monte-Carlo realizations increases, the epistemic uncertainty asymptotically converges and the model standard deviation decreases, because the variability of model parameters is better simulated or explained with a larger sample size. This analysis can quantify the confidence to use fault predictions with less uncertainty. Additionally, the analysis suggests where more training data are needed to reduce the uncertainty in low confidence regions.

scholarly journals PulseNetOne: Fast Unsupervised Pruning of Convolutional Neural Networks for Remote Sensing

2020 ◽  
Vol 12 (7) ◽  
pp. 1092
Author(s):  
David Browne ◽  
Michael Giering ◽  
Steven Prestwich
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Data Augmentation ◽  
Recognition Task ◽  
Scene Recognition ◽  
Training Data ◽  
Learning Approach ◽  
Scene Classification

Scene classification is an important aspect of image/video understanding and segmentation. However, remote-sensing scene classification is a challenging image recognition task, partly due to the limited training data, which causes deep-learning Convolutional Neural Networks (CNNs) to overfit. Another difficulty is that images often have very different scales and orientation (viewing angle). Yet another is that the resulting networks may be very large, again making them prone to overfitting and unsuitable for deployment on memory- and energy-limited devices. We propose an efficient deep-learning approach to tackle these problems. We use transfer learning to compensate for the lack of data, and data augmentation to tackle varying scale and orientation. To reduce network size, we use a novel unsupervised learning approach based on k-means clustering, applied to all parts of the network: most network reduction methods use computationally expensive supervised learning methods, and apply only to the convolutional or fully connected layers, but not both. In experiments, we set new standards in classification accuracy on four remote-sensing and two scene-recognition image datasets.

scholarly journals Nonlinear Chen-Lee Chaotic System Based Deep Convolutional Generative Adversarial Nets for Chatter Diagnosis

2021 ◽  
Author(s):  
Ping-Huan Kuo ◽  
Po-Chien Luan ◽  
Yung-Ruen Tseng ◽  
Her-Terng Yau
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Chaotic System ◽  
Original Data ◽  
Training Data ◽  
Original Training ◽  
Chatter Detection ◽  
Learning Models ◽  
Training Strategy

Abstract Chatter has a direct effect on the precision and life of machine tools and its detection is a crucial issue in all metal machining processes. Traditional methods focus on how to extract discriminative features to help identify chatter. Nowadays, deep learning models have shown an extraordinary ability to extract data features which are their necessary fuel. In this study deep learning models have been substituted for more traditional methods. Chatter data are rare and valuable because the collecting process is extremely difficult. To solve this practical problem an innovative training strategy has been proposed that is combined with a modified convolutional neural network and deep convolutional generative adversarial nets. This improves chatter detection and classification. Convolutional neural networks can be effective chatter classifiers, and adversarial networks can act as generators that produce more data. The convolutional neural networks were trained using original data as well as by forged data produced by the generator. Original training data were collected and preprocessed by the Chen-Lee chaotic system. The adversarial training process used these data to create the generator and the generator could produce enough data to compensate for the lack of training data. The experimental results were compared with without a data generator and data augmentation. The proposed method had an accuracy of 95.3% on leave-one-out cross-validation over ten runs and surpassed other methods and models. The forged data were also compared with original training data as well as data produced by augmentation. The distribution shows that forged data had similar quality and characteristics to the original data. The proposed training strategy provides a high-quality deep learning chatter detection model.

Comparative Performance Evaluation of VGG-16 and Capsnet using Kannada MNIST

2021 ◽  
Vol 9 (8) ◽  
pp. 1190-1194
Author(s):  
Dr. Ramya C
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Character Recognition ◽  
Handwriting Recognition ◽  
Classification Problem ◽  
Training Data ◽  
Comparative Performance ◽  
Regional Languages ◽  
Real Time Application

Abstract: Handwriting recognition is an important problem in character recognition. It is much more difficult especially for regional languages such as Kannada. In this regard there has been a recent surge of interest in designing convolutional neural networks (CNNs) for this problem. However, CNNs typically require large amounts of training data and cannot handle input transformations. Capsule networks, which is referred to as capsNets proposed recently to overcome these shortcomings and posed to revolutionize deep learning solutions. Our particular interest in this work is to recognize kannada digit characters, and making capsnet robust to rotation and transformation. In this paper, we focus to achieve the following objectives :1. Explore whether or not capsnet is capable of providing a better fit for the digit images; 2. Adapt and incorporate capsNets for the problem of kannada MNIST digit classification problem at hand; 3. develop a real time application to take handwritten input from the user and recognize the digit; 4. Compare the capsnet with other models on various parameters. Keywords: Capsule Networks, Deep Learning, Convolutional Neural Networks (CNNs), Kannada MNIST, VGG-16

Classification of Breast Masses on Ultrasound Shear Wave Elastography using Convolutional Neural Networks

2020 ◽  
Vol 42 (4-5) ◽  
pp. 213-220 ◽  
Author(s):  
Tomoyuki Fujioka ◽  
Leona Katsuta ◽  
Kazunori Kubota ◽  
Mio Mori ◽  
Yuka Kikuchi ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Shear Wave ◽  
Convolutional Neural Networks ◽  
Test Data ◽  
Diagnostic Performance ◽  
Shear Wave Elastography ◽  
Training Data ◽  
Breast Masses ◽  
Malignant Breast

We aimed to use deep learning with convolutional neural networks (CNNs) to discriminate images of benign and malignant breast masses on ultrasound shear wave elastography (SWE). We retrospectively gathered 158 images of benign masses and 146 images of malignant masses as training data for SWE. A deep learning model was constructed using several CNN architectures (Xception, InceptionV3, InceptionResNetV2, DenseNet121, DenseNet169, and NASNetMobile) with 50, 100, and 200 epochs. We analyzed SWE images of 38 benign masses and 35 malignant masses as test data. Two radiologists interpreted these test data through a consensus reading using a 5-point visual color assessment (SWEc) and the mean elasticity value (in kPa) (SWEe). Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were calculated. The best CNN model (which was DenseNet169 with 100 epochs), SWEc, and SWEe had a sensitivity of 0.857, 0.829, and 0.914 and a specificity of 0.789, 0.737, and 0.763 respectively. The CNNs exhibited a mean AUC of 0.870 (range, 0.844–0.898), and SWEc and SWEe had an AUC of 0.821 and 0.855. The CNNs had an equal or better diagnostic performance compared with radiologist readings. DenseNet169 with 100 epochs, Xception with 50 epochs, and Xception with 100 epochs had a better diagnostic performance compared with SWEc ( P = 0.018–0.037). Deep learning with CNNs exhibited equal or higher AUC compared with radiologists when discriminating benign from malignant breast masses on ultrasound SWE.

scholarly journals Deep Learning based Effective Steganalysis

2020 ◽  
Vol 9 (4S2) ◽  
pp. 82-85
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Error Detection ◽  
Activation Function ◽  
Sigmoid Function ◽  
Learning Networks ◽  
Feature Maps ◽  
Gating Mechanism

There is an evident paradigm shift in steganalysis techniques with discovery of deep learning networks. As steganalysis is a classification task, it is done by machine learning classifiers and ensembles of them. But with the proliferation of deep learning and Convolutional Neural Networks in many areas, the performance of steganalysis techniques have jumped up to a another high, because of the application of Convolutional Neural Networks. The traditional steganalysis techniques consists two important steps, i.e., feature extraction and classification; where as deep learning networks learn the features automatically, eliminating the need of extraction of handcrafted features. Because of this feature CNNs were highly successful in image recognition and image classification techniques. In addition to that, feature extraction and classification are combined together in deep learning hence classification would be more effective because of the learning of the features which are really important for classification. But in Steganalysis the task is to detect very subtle and weak noise created by the hidden data with steganography techniques. We have designed a deep CNN architecture customized for steganalysis task based on existing residual neural networks frame. We have introduced a descriptor to capture the inter pixel dependencies and which acts as an indicator for weightage of a particular feature maps. Thus the classifier can give more weightage to effective feature maps instead of treating all the feature maps equally. We have also used a gating mechanism by using sigmoid function after nonlinear activation function sandwiched between two fully connected layers. This enhancement to the existing deep residual neural networks has given better results in terms of error detection rate compared to the other deep learning based steganalysis techniques.

scholarly journals Large-Scale Printed Chinese Character Recognition for ID Cards Using Deep Learning and Few Samples Transfer Learning

2021 ◽  
Author(s):  
Yi-Quan Li ◽  
Hao-Sen Chang ◽  
Daw-Tung Lin
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Transfer Learning ◽  
Character Recognition ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Data ◽  
Fine Tuning ◽  
Training Dataset ◽  
Model Parameters

In the field of computer vision, large-scale image classification tasks are both important and highly challenging. With the ongoing advances in deep learning and optical character recognition (OCR) technologies, neural networks designed to perform large-scale classification play an essential role in facilitating OCR systems. In this study, we developed an automatic OCR system designed to identify up to 13,070 large-scale printed Chinese characters by using deep learning neural networks and fine-tuning techniques. The proposed framework comprises four components, including training dataset synthesis and background simulation, image preprocessing and data augmentation, the process of training the model, and transfer learning. The training data synthesis procedure is composed of a character font generation step and a background simulation process. Three background models are proposed to simulate the factors of the background noise and anti-counterfeiting patterns on ID cards. To expand the diversity of the synthesized training dataset, rotation and zooming data augmentation are applied. A massive dataset comprising more than 19.6 million images was thus created to accommodate the variations in the input images and improve the learning capacity of the CNN model. Subsequently, we modified the GoogLeNet neural architecture by replacing the FC layer with a global average pooling layer to avoid overfitting caused by a massive amount of training data. Consequently, the number of model parameters was reduced. Finally, we employed the transfer learning technique to further refine the CNN model using a small number of real data samples. Experimental results show that the overall recognition performance of the proposed approach is significantly better than that of prior methods and thus demonstrate the effectiveness of proposed framework, which exhibited a recognition accuracy as high as 99.39% on the constructed real ID card dataset.

scholarly journals Deep convolutional neural networks for cardiovascular vulnerable plaque detection

2019 ◽  
Vol 277 ◽  
pp. 02024 ◽  
Author(s):  
Lincan Li ◽  
Tong Jia ◽  
Tianqi Meng ◽  
Yizhe Liu
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Vulnerable Plaque ◽  
Recall Rate ◽  
Superior Performance ◽  
Learning Approaches ◽  
Deep Convolutional Neural Networks ◽  
Vulnerable Plaques ◽  
Plaque Detection

In this paper, an accurate two-stage deep learning method is proposed to detect vulnerable plaques in ultrasonic images of cardiovascular. Firstly, a Fully Convonutional Neural Network (FCN) named U-Net is used to segment the original Intravascular Optical Coherence Tomography (IVOCT) cardiovascular images. We experiment on different threshold values to find the best threshold for removing noise and background in the original images. Secondly, a modified Faster RCNN is adopted to do precise detection. The modified Faster R-CNN utilize six-scale anchors (122,162,322,642,1282,2562) instead of the conventional one scale or three scale approaches. First, we present three problems in cardiovascular vulnerable plaque diagnosis, then we demonstrate how our method solve these problems. The proposed method in this paper apply deep convolutional neural networks to the whole diagnostic procedure. Test results show the Recall rate, Precision rate, IoU (Intersection-over-Union) rate and Total score are 0.94, 0.885, 0.913 and 0.913 respectively, higher than the 1st team of CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge. AP of the designed Faster RCNN is 83.4%, higher than conventional approaches which use one-scale or three-scale anchors. These results demonstrate the superior performance of our proposed method and the power of deep learning approaches in diagnose cardiovascular vulnerable plaques.

scholarly journals Deep Malaria Parasite Detection in Thin Blood Smear Microscopic Images

2021 ◽  
Vol 11 (5) ◽  
pp. 2284
Author(s):  
Asma Maqsood ◽  
Muhammad Shahid Farid ◽  
Muhammad Hassan Khan ◽  
Marcin Grzegorzek
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Red Blood Cells ◽  
Convolutional Neural Networks ◽  
Blood Cells ◽  
Superior Performance ◽  
Learning Models ◽  
Infected Female ◽  
The World ◽  
Augmentation Techniques

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

Sign in / Sign up

Export Citation Format

Share Document