Two-dimensional deep learning inversion of magnetotelluric sounding data

Abstract Conventional linear iterative methods for magnetotelluric sounding (MT) suffer from considerable limitations related to difficulties in selecting the initial model and local optima. On the other hand, conventional intelligent nonlinear methods exhibit slow convergence and low accuracy. In this study, we propose an inversion strategy based on the deep learning (DL) deep belief network (DBN) to realise the instantaneous inversion of MT data. A scaled momentum learning rate is introduced to improve the convergence performance of the restricted Boltzmann machine during the DBN pre-training stage, and a novel activation function (DSoft) is introduced to enhance the global optimisation capability during the DBN fine-tuning stage. To address the difficulty in designing the sample data when prior information is lacking, we employ the k-means++ algorithm to cluster the MT field data and use the clustering results as the prior information to guide the construction of the sample dataset. Then, based on the proposed DBN, we ensure end-to-end mapping directly from the apparent resistivity to the resistivity model. We implement two groups of experiments to demonstrate the validity of both improvements on the DBN. We consider six types of geoelectric model from the test set to demonstrate the feasibility and effectiveness of the proposed DBN method for MT 2D inversion, which we further compare with the well-known least-square regularisation method for several extended geoelectric models and field data. The qualitative and quantitative analyses show that the DL inversion method is promising as it can accurately delineate the subsurface structures and perform rapid inversion.

Download Full-text

Application of Deep Learning Technique to Predict Downhole Pressure Differential in Eccentric Annulus of Ultra-Deep Well

10.1115/omae2021-62621 ◽

2021 ◽

Author(s):

Shwetank Krishna ◽

Syahrir Ridha ◽

Suhaib Umer Ilyas ◽

Scott Campbell ◽

Uday Bhan ◽

...

Keyword(s):

Deep Learning ◽

Pressure Gradient ◽

Field Data ◽

Activation Function ◽

Pressure Differential ◽

Optimal Arrangement ◽

Eccentric Annulus ◽

Learning Framework ◽

Deep Well ◽

Deep Wells

Abstract Accurate prediction of downhole pressure differential (surge/swab pressure gradient) in the eccentric annulus of ultra-deep wells during tripping operation is a necessity to optimize well geometry, reduction of drilling anomalies, and prevention of hazardous drilling accidents. Therefore, a new predictive model is developed to forecast surge/swab pressure gradient by using feed-forward and backpropagation deep neural networks (FFBP-DNN). A theoretical-based model is developed that follows the physical and mechanical aspects of surge/swab pressure generation in eccentric annulus during tripping operation. The data generated from this model, field data, and experimental data are used to train and test the FFBP-DNN networks. The network is developed used Keras’s deep learning framework. After testing the models, the most optimal arrangement of FFBP-DNN is the ReLU algorithm as an activation function, 4-hidden layers, the learning rate of 0.003, and 2300 of training numbers. The optimum FFBP-DNN model is validated by comparing it with field data (Wells K 470 and K 480, North Sea). It shows an excellent argument between predicted data and field data with an error range of ±7.68 %.

Download Full-text

Deep Neural Network for Multi-Class Prediction of Student Performance in Educational Data

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.b2155.078219 ◽

2019 ◽

Vol 8 (2) ◽

pp. 5073-5081

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Student Performance ◽

Activation Function ◽

Machine Learning Algorithms ◽

Training Data ◽

Fine Tuning ◽

Academic Excellence ◽

Data Set

Prediction of student performance is the significant part in processing the educational data. Machine learning algorithms are leading the role in this process. Deep learning is one of the important concepts of machine learning algorithm. In this paper, we applied the deep learning technique for prediction of the academic excellence of the students using R Programming. Keras and Tensorflow libraries utilized for making the model using neural network on the Kaggle dataset. The data is separated into testing data training data set. Plot the neural network model using neuralnet method and created the Deep Learning model using two hidden layers using ReLu activation function and one output layer using softmax activation function. After fine tuning process until the stable changes; this model produced accuracy as 85%.

Download Full-text

Stacked Denoise Autoencoder Based Feature Extraction and Classification for Hyperspectral Images

Journal of Sensors ◽

10.1155/2016/3632943 ◽

2016 ◽

Vol 2016 ◽

pp. 1-10 ◽

Cited By ~ 65

Author(s):

Chen Xing ◽

Li Ma ◽

Xiaoquan Yang

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

Activation Function ◽

Hyperspectral Data ◽

Fine Tuning ◽

Support Vector ◽

Svm Classifier ◽

Nonlinear Properties ◽

Feature Representations ◽

High Level

Deep learning methods have been successfully applied to learn feature representations for high-dimensional data, where the learned features are able to reveal the nonlinear properties exhibited in the data. In this paper, deep learning method is exploited for feature extraction of hyperspectral data, and the extracted features can provide good discriminability for classification task. Training a deep network for feature extraction and classification includes unsupervised pretraining and supervised fine-tuning. We utilized stacked denoise autoencoder (SDAE) method to pretrain the network, which is robust to noise. In the top layer of the network, logistic regression (LR) approach is utilized to perform supervised fine-tuning and classification. Since sparsity of features might improve the separation capability, we utilized rectified linear unit (ReLU) as activation function in SDAE to extract high level and sparse features. Experimental results using Hyperion, AVIRIS, and ROSIS hyperspectral data demonstrated that the SDAE pretraining in conjunction with the LR fine-tuning and classification (SDAE_LR) can achieve higher accuracies than the popular support vector machine (SVM) classifier.

Download Full-text

Matching Large Baseline Oblique Stereo Images Using an End-to-End Convolutional Neural Network

Remote Sensing ◽

10.3390/rs13020274 ◽

2021 ◽

Vol 13 (2) ◽

pp. 274

Author(s):

Guobiao Yao ◽

Alper Yilmaz ◽

Li Zhang ◽

Fei Meng ◽

Haibin Ai ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Stereo Matching ◽

Least Square ◽

Affine Invariant ◽

Stereo Images ◽

Distance Ratio ◽

Matching Algorithm ◽

End To End

The available stereo matching algorithms produce large number of false positive matches or only produce a few true-positives across oblique stereo images with large baseline. This undesired result happens due to the complex perspective deformation and radiometric distortion across the images. To address this problem, we propose a novel affine invariant feature matching algorithm with subpixel accuracy based on an end-to-end convolutional neural network (CNN). In our method, we adopt and modify a Hessian affine network, which we refer to as IHesAffNet, to obtain affine invariant Hessian regions using deep learning framework. To improve the correlation between corresponding features, we introduce an empirical weighted loss function (EWLF) based on the negative samples using K nearest neighbors, and then generate deep learning-based descriptors with high discrimination that is realized with our multiple hard network structure (MTHardNets). Following this step, the conjugate features are produced by using the Euclidean distance ratio as the matching metric, and the accuracy of matches are optimized through the deep learning transform based least square matching (DLT-LSM). Finally, experiments on Large baseline oblique stereo images acquired by ground close-range and unmanned aerial vehicle (UAV) verify the effectiveness of the proposed approach, and comprehensive comparisons demonstrate that our matching algorithm outperforms the state-of-art methods in terms of accuracy, distribution and correct ratio. The main contributions of this article are: (i) our proposed MTHardNets can generate high quality descriptors; and (ii) the IHesAffNet can produce substantial affine invariant corresponding features with reliable transform parameters.

Download Full-text

Sparse Autoencoder-based Multi-head Deep Neural Networks for Machinery Fault Diagnostics with Detection of Novelties

Chinese Journal of Mechanical Engineering ◽

10.1186/s10033-021-00569-0 ◽

2021 ◽

Vol 34 (1) ◽

Author(s):

Zhe Yang ◽

Dejan Gjorgjevikj ◽

Jianyu Long ◽

Yanyang Zi ◽

Shaohui Zhang ◽

...

Keyword(s):

Fault Diagnosis ◽

Supervised Classification ◽

Deep Neural Networks ◽

Activation Function ◽

Reconstruction Error ◽

Fine Tuning ◽

Fault Diagnostics ◽

Sparse Autoencoder

AbstractSupervised fault diagnosis typically assumes that all the types of machinery failures are known. However, in practice unknown types of defect, i.e., novelties, may occur, whose detection is a challenging task. In this paper, a novel fault diagnostic method is developed for both diagnostics and detection of novelties. To this end, a sparse autoencoder-based multi-head Deep Neural Network (DNN) is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data. The detection of novelties is based on the reconstruction error. Moreover, the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function, instead of performing the pre-training and fine-tuning phases required for classical DNNs. The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer. The results show that its performance is satisfactory both in detection of novelties and fault diagnosis, outperforming other state-of-the-art methods. This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect, but also detect unknown types of defects.

Download Full-text

Deep Learning-Based Differentiation between Mucinous Cystic Neoplasm and Serous Cystic Neoplasm in the Pancreas Using Endoscopic Ultrasonography

Diagnostics ◽

10.3390/diagnostics11061052 ◽

2021 ◽

Vol 11 (6) ◽

pp. 1052

Author(s):

Leang Sim Nguon ◽

Kangwon Seo ◽

Jung-Hyun Lim ◽

Tae-Jun Song ◽

Sung-Hyun Cho ◽

...

Keyword(s):

Decision Making ◽

Deep Learning ◽

Network Model ◽

Endoscopic Ultrasonography ◽

Data Augmentation ◽

Clinical Information ◽

Training Data ◽

Fine Tuning ◽

Cystic Neoplasm ◽

Cystic Neoplasms

Mucinous cystic neoplasms (MCN) and serous cystic neoplasms (SCN) account for a large portion of solitary pancreatic cystic neoplasms (PCN). In this study we implemented a convolutional neural network (CNN) model using ResNet50 to differentiate between MCN and SCN. The training data were collected retrospectively from 59 MCN and 49 SCN patients from two different hospitals. Data augmentation was used to enhance the size and quality of training datasets. Fine-tuning training approaches were utilized by adopting the pre-trained model from transfer learning while training selected layers. Testing of the network was conducted by varying the endoscopic ultrasonography (EUS) image sizes and positions to evaluate the network performance for differentiation. The proposed network model achieved up to 82.75% accuracy and a 0.88 (95% CI: 0.817–0.930) area under curve (AUC) score. The performance of the implemented deep learning networks in decision-making using only EUS images is comparable to that of traditional manual decision-making using EUS images along with supporting clinical information. Gradient-weighted class activation mapping (Grad-CAM) confirmed that the network model learned the features from the cyst region accurately. This study proves the feasibility of diagnosing MCN and SCN using a deep learning network model. Further improvement using more datasets is needed.

Download Full-text

Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones

Scientific Reports ◽

10.1038/s41598-021-92776-x ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Andrew P. Creagh ◽

Florian Lipsmeier ◽

Michael Lindemann ◽

Maarten De Vos

Keyword(s):

Multiple Sclerosis ◽

Deep Learning ◽

Inertial Sensor ◽

Heterogeneous Data ◽

Fine Tuning ◽

Sensor Data ◽

Support Vector ◽

Deep Convolutional Neural Networks ◽

Healthcare Applications ◽

Feature Based

AbstractThe emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8–15%. A lack of transparency of “black-box” deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.

Download Full-text

Inverse design of layered periodic wave barriers based on deep learning

Proceedings of the Institution of Mechanical Engineers Part L Journal of Materials Design and Applications ◽

10.1177/14644207211016886 ◽

2021 ◽

pp. 146442072110168

Author(s):

Chen-Xu Liu ◽

Gui-Lan Yu

Keyword(s):

Deep Learning ◽

Activation Function ◽

Periodic Wave ◽

P Wave ◽

Soil Parameters ◽

S Wave ◽

Output Frequency ◽

Great Generality ◽

Typical Range ◽

Deep Learning Model

This study presents an approach based on deep learning to design layered periodic wave barriers with consideration of typical range of soil parameters. Three cases are considered where P wave and S wave exist separately or simultaneously. The deep learning model is composed of an autoencoder with a pretrained decoder which has three branches to output frequency attenuation domains for three different cases. A periodic activation function is used to improve the design accuracy, and condition variables are applied in the code layer of the autoencoder to meet the requirements of practical multi working conditions. Forty thousand sets of data are generated to train, validate, and test the model, and the designed results are highly consistent with the targets. The presented approach has great generality, feasibility, rapidity, and accuracy on designing layered periodic wave barriers which exhibit good performance in wave suppression in targeted frequency range.

Download Full-text

Deep Learning Representation from Electroencephalography of Early-Stage Creutzfeldt-Jakob Disease and Features for Differentiation from Rapidly Progressive Dementia

International Journal of Neural Systems ◽

10.1142/s0129065716500398 ◽

2016 ◽

Vol 27 (02) ◽

pp. 1650039 ◽

Cited By ~ 57

Author(s):

Francesco Carlo Morabito ◽

Maurizio Campolo ◽

Nadia Mammone ◽

Mario Versaci ◽

Silvana Franceschetti ◽

...

Keyword(s):

Deep Learning ◽

Supervised Learning ◽

Early Stage ◽

Processing System ◽

Fine Tuning ◽

Permutation Entropy ◽

Support Vector ◽

Progressive Dementia ◽

Time Frequency ◽

Jakob Disease

A novel technique of quantitative EEG for differentiating patients with early-stage Creutzfeldt–Jakob disease (CJD) from other forms of rapidly progressive dementia (RPD) is proposed. The discrimination is based on the extraction of suitable features from the time-frequency representation of the EEG signals through continuous wavelet transform (CWT). An average measure of complexity of the EEG signal obtained by permutation entropy (PE) is also included. The dimensionality of the feature space is reduced through a multilayer processing system based on the recently emerged deep learning (DL) concept. The DL processor includes a stacked auto-encoder, trained by unsupervised learning techniques, and a classifier whose parameters are determined in a supervised way by associating the known category labels to the reduced vector of high-level features generated by the previous processing blocks. The supervised learning step is carried out by using either support vector machines (SVM) or multilayer neural networks (MLP-NN). A subset of EEG from patients suffering from Alzheimer’s Disease (AD) and healthy controls (HC) is considered for differentiating CJD patients. When fine-tuning the parameters of the global processing system by a supervised learning procedure, the proposed system is able to achieve an average accuracy of 89%, an average sensitivity of 92%, and an average specificity of 89% in differentiating CJD from RPD. Similar results are obtained for CJD versus AD and CJD versus HC.

Download Full-text

REAL-TIME 2.5D INVERSION OF LWD RESISTIVITY MEASUREMENTS USING DEEP LEARNING FOR GEOSTEERING APPLICATIONS ACROSS FAULTED FORMATIONS

10.30632/spwla-2021-0104 ◽

2021 ◽

Author(s):

Kyubo Noh ◽

◽

Carlos Torres-Verdín ◽

David Pardo ◽

◽

...

Keyword(s):

Deep Learning ◽

Real Time ◽

Fault Plane ◽

Synthetic Data ◽

Field Measurements ◽

Geological Structure ◽

Inversion Method ◽

Adaptive Finite Element ◽

Resistivity Measurements ◽

Lwd Resistivity

We develop a Deep Learning (DL) inversion method for the interpretation of 2.5-dimensional (2.5D) borehole resistivity measurements that requires negligible online computational costs. The method is successfully verified with the inversion of triaxial LWD resistivity measurements acquired across faulted and anisotropic formations. Our DL inversion workflow employs four independent DL architectures. The first one identifies the type of geological structure among several predefined types. Subsequently, the second, third, and fourth architectures estimate the corresponding spatial resistivity distributions that are parameterized (1) without the crossings of bed boundaries or fault plane, (2) with the crossing of a bed boundary but without the crossing of a fault plane, and (3) with the crossing of the fault plane, respectively. Each DL architecture employs convolutional layers and is trained with synthetic data obtained from an accurate high-order, mesh-adaptive finite-element forward numerical simulator. Numerical results confirm the importance of using multi-component resistivity measurements -specifically cross-coupling resistivity components- for the successful reconstruction of 2.5D resistivity distributions adjacent to the well trajectory. The feasibility and effectiveness of the developed inversion workflow is assessed with two synthetic examples inspired by actual field measurements. Results confirm that the proposed DL method successfully reconstructs 2.5D resistivity distributions, location and dip angles of bed boundaries, and the location of the fault plane, and is therefore reliable for real-time well geosteering applications.

Download Full-text