Intelligent random noise modeling by the improved variational autoencoding method and its application to data augmentation

Geophysics ◽  
2021 ◽  
Vol 86 (1) ◽  
pp. T19-T31
Author(s):  
Qiankun Feng ◽  
Yue Li ◽  
Hongzhou Wang
Keyword(s):  
Seismic Data ◽  
Data Augmentation ◽  
Random Noise ◽  
Gaussian Mixture ◽  
Training Data ◽  
Training Set ◽  
Noise Modeling ◽  
Noise Data ◽  
Learning Parameter ◽  
Data Processing Methods

Deep-learning methods facilitate the development of seismic data processing methods; however, they also offer some challenges. The primary challenges are the lack of labeled samples for training, due to heterogeneity in seismic data, expensive acquisition apparatus, and data confidentiality. These problems limit the acquisition of high-quality training data. To solve this problem, we have developed variational autoencoding (VAE) to generate synthetic noise for data augmentation; however, the simplified Kullback-Leibler (KL) distance definition and parameter learning result in the outputs of the original VAE being blurry. To optimize VAE for simulating random desert noise and improve its simulation capability, here we have developed an improved VAE based on KL redefinition and learning parameter replacement. Specifically, we (1) build a training set containing desert random noise samples, (2) redefine the KL distance calculated between two Gaussian mixture densities (rather than two simple Gaussians) because the KL distance plays an important role in the learning accuracy of VAE, and (3) use [Formula: see text] rather than [Formula: see text] to improve the learning efficiency. Statistical analysis indicates that the simulated random noise is statistically indistinguishable from real noise, indicating that our improved VAE is suitable for noise modeling. We also trained a denoising convolutional neural network (DnCNN) using the simulated noise. Data augmentation conducted using the simulated noise improved the effect of DnCNN, proving that our method contributes to data augmentation.

Attenuation of random noise using denoising convolutional neural networks

2019 ◽  
Vol 7 (3) ◽  
pp. SE269-SE280
Author(s):  
Xu Si ◽  
Yijun Yuan ◽  
Tinghua Si ◽  
Shiwen Gao
Keyword(s):  
Neural Network ◽  
Data Processing ◽  
Seismic Data ◽  
Random Noise ◽  
Training Data ◽  
Prediction Errors ◽  
Network Learning ◽  
Deep Network ◽  
Residual Image ◽  
And Training

Random noise often contaminates seismic data and reduces its signal-to-noise ratio. Therefore, the removal of random noise has been an essential step in seismic data processing. The [Formula: see text]-[Formula: see text] predictive filtering method is one of the most widely used methods in suppressing random noise. However, when the subsurface structure becomes complex, this method suffers from higher prediction errors owing to the large number of different dip components that need to be predicted. Here, we used a denoising convolutional neural network (DnCNN) algorithm to attenuate random noise in seismic data. This method does not assume the linearity and stationarity of the signal in the conventional [Formula: see text]-[Formula: see text] domain prediction technique, and it involves creating a set of training data that are obtained by data processing, feeding the neural network with the training data obtained, and deep network learning and training. During deep network learning and training, the activation function and batch normalization are used to solve the gradient vanishing and gradient explosion problems, and the residual learning technique is used to improve the calculation precision, respectively. After finishing deep network learning and training, the network will have the ability to separate the residual image from the seismic data with noise. Then, clean images can be obtained by subtracting the residual image from the raw data with noise. Tests on the synthetic and real data demonstrate that the DnCNN algorithm is very effective for random noise attenuation in seismic data.

scholarly journals Convolutional Neural Network-Based Discriminator for Outlier Detection

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Fahad Alharbi ◽  
Khalil El Hindi ◽  
Saad Al Ahmadi ◽  
Hussien Alsalamn
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Systematic Approach ◽  
Data Augmentation ◽  
Random Noise ◽  
Training Data ◽  
Human Errors ◽  
Machine Learning Methods ◽  
Benchmark Datasets ◽  
Trusted Data

Noise in training data increases the tendency of many machine learning methods to overfit the training data, which undermines the performance. Outliers occur in big data as a result of various factors, including human errors. In this work, we present a novel discriminator model for the identification of outliers in the training data. We propose a systematic approach for creating training datasets to train the discriminator based on a small number of genuine instances (trusted data). The noise discriminator is a convolutional neural network (CNN). We evaluate the discriminator’s performance using several benchmark datasets and with different noise ratios. We inserted random noise in each dataset and trained discriminators to clean them. Different discriminators were trained using different numbers of genuine instances with and without data augmentation. We compare the performance of the proposed noise-discriminator method with seven other methods proposed in the literature using several benchmark datasets. Our empirical results indicate that the proposed method is very competitive to the other methods. It actually outperforms them for pair noise.

scholarly journals A Natural Images Pre-Trained Deep Learning Method for Seismic Random Noise Attenuation

2022 ◽  
Vol 14 (2) ◽  
pp. 263
Author(s):  
Haixia Zhao ◽  
Tingting Bai ◽  
Zhiqiang Wang
Keyword(s):  
Deep Learning ◽  
Seismic Data ◽  
Field Data ◽  
Noise Suppression ◽  
Signal To Noise Ratio ◽  
Random Noise ◽  
Natural Images ◽  
Training Data ◽  
Learning Approaches ◽  
Learning Method

Seismic field data are usually contaminated by random or complex noise, which seriously affect the quality of seismic data contaminating seismic imaging and seismic interpretation. Improving the signal-to-noise ratio (SNR) of seismic data has always been a key step in seismic data processing. Deep learning approaches have been successfully applied to suppress seismic random noise. The training examples are essential in deep learning methods, especially for the geophysical problems, where the complete training data are not easy to be acquired due to high cost of acquisition. In this work, we propose a natural images pre-trained deep learning method to suppress seismic random noise through insight of the transfer learning. Our network contains pre-trained and post-trained networks: the former is trained by natural images to obtain the preliminary denoising results, while the latter is trained by a small amount of seismic images to fine-tune the denoising effects by semi-supervised learning to enhance the continuity of geological structures. The results of four types of synthetic seismic data and six field data demonstrate that our network has great performance in seismic random noise suppression in terms of both quantitative metrics and intuitive effects.

Complete and representative training of neural networks: A generalization study using double noise injection and natural images

Geophysics ◽  
2021 ◽  
pp. 1-43
Author(s):  
Chao Zhang ◽  
Mirko van der Baan
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Seismic Data ◽  
Learning Algorithm ◽  
Random Noise ◽  
Solution Space ◽  
Natural Images ◽  
Training Data ◽  
Noisy Input ◽  
Noise Injection

Neural networks hold substantial promise to automate various processing and interpretation tasks. Yet their performance is often sub-optimal compared with standard but more closely guided approaches. Lack of performance is often attributed to poor generalization, in particular if fewer training examples are provided than free parameters exist in the machine learning algorithm. In this case the training data are typically memorized instead of the algorithm learning the underlying general trends. Network generalization is improved if the provided samples are representative, in that they describe all features of interest well. We argue that a more subtle condition preventing poor performance is that the provided examples must also be complete; the examples must span the full solution space. Ensuring completeness during training is challenging unless the target application is well understood. We illustrate that one possible solution is to make the problem more general if this greatly increases the number of available training data. For instance, if seismic images are treated as a subclass of natural images, then a deep-learning-based denoiser for seismic data can be trained using exclusively natural images. The latter are widely available. The resulting denoising algorithm has never seen any seismic data during the training stage; yet it displays a performance comparable to standard and advanced random-noise reduction methods. We exclude any seismic data during training to demonstrate the natural images are both complete and representative for this specific task. Furthermore, we apply a novel approach to increase the amount of training data known as double noise injection, providing both noisy input and output images during the training process. Given the importance of network generalization, we hope that insights gained in this study may help improve the performance of a range of machine learning applications in geophysics.

A method for adequate selection of training data sets to reconstruct seismic field data using a convolutional U-Net

Geophysics ◽  
2021 ◽  
pp. 1-103
Author(s):  
Jiho Park ◽  
Jihun Choi ◽  
Soon Jee Seol ◽  
Joongmoo Byun ◽  
Young Kim
Keyword(s):  
Data Analysis ◽  
Seismic Data ◽  
Field Data ◽  
Training Data ◽  
Data Reconstruction ◽  
Data Sets ◽  
Training Set ◽  
Data Set ◽  
Seismic Data Reconstruction ◽  
Target Data

Deep learning (DL) methods are recently introduced for seismic signal processing. Using DL methods, many researchers have adopted these novel techniques in an attempt to construct a DL model for seismic data reconstruction. The performance of DL-based methods depends heavily on what is learned from the training data. We focus on constructing the DL model that well reflect the features of target data sets. The main goal is to integrate DL with an intuitive data analysis approach that compares similar patterns prior to the DL training stage. We have developed a two-sequential method consisting of two stage: (i) analyzing training and target data sets simultaneously for determining target-informed training set and (ii) training the DL model with this training data set to effectively interpolate the seismic data. Here, we introduce the convolutional autoencoder t-distributed stochastic neighbor embedding (CAE t-SNE) analysis that can provide the insight into the results of interpolation through the analysis of both the training and target data sets prior to DL model training. The proposed method were tested with synthetic and field data. Dense seismic gathers (e.g. common-shot gathers; CSGs) were used as a labeled training data set, and relatively sparse seismic gather (e.g. common-receiver gathers; CRGs) were reconstructed in both cases. The reconstructed results and SNRs demonstrated that the training data can be efficiently selected using CAE t-SNE analysis and the spatial aliasing of CRGs was successfully alleviated by the trained DL model with this training data, which contain target features. These results imply that the data analysis for selecting target-informed training set is very important for successful DL interpolation. Additionally, the proposed analysis method can also be applied to investigate the similarities between training and target data sets for another DL-based seismic data reconstruction tasks.

scholarly journals A Contrast Augmentation Approach to Improve Multi-Scanner Generalization in MRI

2021 ◽  
Vol 15 ◽  
Author(s):  
Maria Ines Meyer ◽  
Ezequiel de la Rosa ◽  
Nuno Pedrosa de Barros ◽  
Roberto Paolella ◽  
Koen Van Leemput ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Brain Mri ◽  
Gaussian Mixture Models ◽  
Brain Magnetic Resonance Imaging ◽  
Bias Field ◽  
Brain Structures ◽  
Gaussian Mixture ◽  
Training Data ◽  
Training Dataset ◽  
Clinical Scenarios

Most data-driven methods are very susceptible to data variability. This problem is particularly apparent when applying Deep Learning (DL) to brain Magnetic Resonance Imaging (MRI), where intensities and contrasts vary due to acquisition protocol, scanner- and center-specific factors. Most publicly available brain MRI datasets originate from the same center and are homogeneous in terms of scanner and used protocol. As such, devising robust methods that generalize to multi-scanner and multi-center data is crucial for transferring these techniques into clinical practice. We propose a novel data augmentation approach based on Gaussian Mixture Models (GMM-DA) with the goal of increasing the variability of a given dataset in terms of intensities and contrasts. The approach allows to augment the training dataset such that the variability in the training set compares to what is seen in real world clinical data, while preserving anatomical information. We compare the performance of a state-of-the-art U-Net model trained for segmenting brain structures with and without the addition of GMM-DA. The models are trained and evaluated on single- and multi-scanner datasets. Additionally, we verify the consistency of test-retest results on same-patient images (same and different scanners). Finally, we investigate how the presence of bias field influences the performance of a model trained with GMM-DA. We found that the addition of the GMM-DA improves the generalization capability of the DL model to other scanners not present in the training data, even when the train set is already multi-scanner. Besides, the consistency between same-patient segmentation predictions is improved, both for same-scanner and different-scanner repetitions. We conclude that GMM-DA could increase the transferability of DL models into clinical scenarios.

Repeatability enhancement of time-lapse seismic data via a convolutional autoencoder

2021 ◽  
Author(s):  
Hyunggu Jun ◽  
Yongchae Cho
Keyword(s):  
Machine Learning ◽  
Seismic Data ◽  
Network Architecture ◽  
Numerical Experiments ◽  
Data Augmentation ◽  
Proper Time ◽  
Time Lapse ◽  
Training Data ◽  
Convolutional Autoencoder ◽  
Target Data

Summary In an ideal case, the time-lapse differences in 4D seismic data should only reflect the changes of the subsurface geology. Practically, however, undesirable discrepancies are generated because of various reasons. Therefore, proper time-lapse processing techniques are required to improve the repeatability of time-lapse seismic data and to capture accurate seismic information to analyze target changes. In this study, we propose a machine learning-based time-lapse seismic data processing method improving repeatability. A training data construction method, training strategy, and machine learning network architecture based on a convolutional autoencoder are proposed. Uniform manifold approximation and projection are applied to the training and target data to analyze the features corresponding to each data point. When the feature distribution of the training data is different from the target data, we implement data augmentation to enhance the diversity of the training data. The method is verified through numerical experiments using both synthetic and field time-lapse seismic data, and the results are analyzed with several methods, including a comparison of repeatability metrics. From the results of the numerical experiments, we can conclude that the proposed convolutional autoencoder can enhance the repeatability of the time-lapse seismic data and increase the accuracy of observed variations in seismic signals generated from target changes.

scholarly journals Building Damage Detection from Post-Event Aerial Imagery Using Single Shot Multibox Detector

2019 ◽  
Vol 9 (6) ◽  
pp. 1128 ◽  
Author(s):  
Yundong Li ◽  
Wei Hu ◽  
Han Dong ◽  
Xueyan Zhang
Keyword(s):  
Machine Learning ◽  
Data Augmentation ◽  
Hurricane Sandy ◽  
Training Data ◽  
Aerial Images ◽  
Detection Methods ◽  
Single Shot ◽  
Data Set ◽  
Augmentation Strategies ◽  
Post Disaster

Using aerial cameras, satellite remote sensing or unmanned aerial vehicles (UAV) equipped with cameras can facilitate search and rescue tasks after disasters. The traditional manual interpretation of huge aerial images is inefficient and could be replaced by machine learning-based methods combined with image processing techniques. Given the development of machine learning, researchers find that convolutional neural networks can effectively extract features from images. Some target detection methods based on deep learning, such as the single-shot multibox detector (SSD) algorithm, can achieve better results than traditional methods. However, the impressive performance of machine learning-based methods results from the numerous labeled samples. Given the complexity of post-disaster scenarios, obtaining many samples in the aftermath of disasters is difficult. To address this issue, a damaged building assessment method using SSD with pretraining and data augmentation is proposed in the current study and highlights the following aspects. (1) Objects can be detected and classified into undamaged buildings, damaged buildings, and ruins. (2) A convolution auto-encoder (CAE) that consists of VGG16 is constructed and trained using unlabeled post-disaster images. As a transfer learning strategy, the weights of the SSD model are initialized using the weights of the CAE counterpart. (3) Data augmentation strategies, such as image mirroring, rotation, Gaussian blur, and Gaussian noise processing, are utilized to augment the training data set. As a case study, aerial images of Hurricane Sandy in 2012 were maximized to validate the proposed method’s effectiveness. Experiments show that the pretraining strategy can improve of 10% in terms of overall accuracy compared with the SSD trained from scratch. These experiments also demonstrate that using data augmentation strategies can improve mAP and mF1 by 72% and 20%, respectively. Finally, the experiment is further verified by another dataset of Hurricane Irma, and it is concluded that the paper method is feasible.

scholarly journals Automated Ventricular System Segmentation in Paediatric Patients Treated for Hydrocephalus Using Deep Learning Methods

2019 ◽  
Vol 2019 ◽  
pp. 1-9 ◽  
Author(s):  
Michał Klimont ◽  
Mateusz Flieger ◽  
Jacek Rzeszutek ◽  
Joanna Stachera ◽  
Aleksandra Zakrzewska ◽  
...  
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Ct Images ◽  
Policy Transfer ◽  
Training Data ◽  
Intraobserver Variability ◽  
Practical Applications ◽  
Brain Scans ◽  
Rate Policy ◽  
Ct Brain

Hydrocephalus is a common neurological condition that can have traumatic ramifications and can be lethal without treatment. Nowadays, during therapy radiologists have to spend a vast amount of time assessing the volume of cerebrospinal fluid (CSF) by manual segmentation on Computed Tomography (CT) images. Further, some of the segmentations are prone to radiologist bias and high intraobserver variability. To improve this, researchers are exploring methods to automate the process, which would enable faster and more unbiased results. In this study, we propose the application of U-Net convolutional neural network in order to automatically segment CT brain scans for location of CSF. U-Net is a neural network that has proven to be successful for various interdisciplinary segmentation tasks. We optimised training using state of the art methods, including “1cycle” learning rate policy, transfer learning, generalized dice loss function, mixed float precision, self-attention, and data augmentation. Even though the study was performed using a limited amount of data (80 CT images), our experiment has shown near human-level performance. We managed to achieve a 0.917 mean dice score with 0.0352 standard deviation on cross validation across the training data and a 0.9506 mean dice score on a separate test set. To our knowledge, these results are better than any known method for CSF segmentation in hydrocephalic patients, and thus, it is promising for potential practical applications.

scholarly journals GPR B-Scan Image Denoising via Multi-Scale Convolutional Autoencoder with Data Augmentation

Electronics ◽  
2021 ◽  
Vol 10 (11) ◽  
pp. 1269
Author(s):  
Jiabin Luo ◽  
Wentai Lei ◽  
Feifei Hou ◽  
Chenghao Wang ◽  
Qiang Ren ◽  
...  
Keyword(s):  
Image Denoising ◽  
Data Augmentation ◽  
Noise Suppression ◽  
Random Noise ◽  
Similarity Index ◽  
Structural Similarity ◽  
Training Dataset ◽  
Generative Adversarial Network ◽  
Multi Scale ◽  
Convolutional Autoencoder

Ground-penetrating radar (GPR), as a non-invasive instrument, has been widely used in civil engineering. In GPR B-scan images, there may exist random noise due to the influence of the environment and equipment hardware, which complicates the interpretability of the useful information. Many methods have been proposed to eliminate or suppress the random noise. However, the existing methods have an unsatisfactory denoising effect when the image is severely contaminated by random noise. This paper proposes a multi-scale convolutional autoencoder (MCAE) to denoise GPR data. At the same time, to solve the problem of training dataset insufficiency, we designed the data augmentation strategy, Wasserstein generative adversarial network (WGAN), to increase the training dataset of MCAE. Experimental results conducted on both simulated, generated, and field datasets demonstrated that the proposed scheme has promising performance for image denoising. In terms of three indexes: the peak signal-to-noise ratio (PSNR), the time cost, and the structural similarity index (SSIM), the proposed scheme can achieve better performance of random noise suppression compared with the state-of-the-art competing methods (e.g., CAE, BM3D, WNNM).

Sign in / Sign up

Export Citation Format

Share Document