Regularization strategies for deep-learning-based salt model building

2019 ◽  
Vol 7 (4) ◽  
pp. T911-T922
Author(s):  
Satyakee Sen ◽  
Sribharath Kainkaryam ◽  
Cen Ong ◽  
Arvind Sharma
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Model Building ◽  
Ground Truth ◽  
Real Data ◽  
Test Time ◽  
Generalization Error ◽  
Training Set ◽  
Production Scale ◽  
Ensemble Strategy

Salt model building has long been considered a severe bottleneck for large-scale 3D seismic imaging projects. It is one of the most time-consuming, labor-intensive, and difficult-to-automate processes in the entire depth imaging workflow requiring significant intervention by domain experts to manually interpret the salt bodies on noisy, low-frequency, and low-resolution seismic images at each iteration of the salt model building process. The difficulty and need for automating this task is well-recognized by the imaging community and has propelled the use of deep-learning-based convolutional neural network (CNN) architectures to carry out this task. However, significant challenges remain for reliable production-scale deployment of CNN-based methods for salt model building. This is mainly due to the poor generalization capabilities of these networks. When used on new surveys, never seen by the CNN models during the training stage, the interpretation accuracy of these models drops significantly. To remediate this key problem, we have introduced a U-shaped encoder-decoder type CNN architecture trained using a specialized regularization strategy aimed at reducing the generalization error of the network. Our regularization scheme perturbs the ground truth labels in the training set. Two different perturbations are discussed: one that randomly changes the labels of the training set, flipping salt labels to sediments and vice versa and the second that smooths the labels. We have determined that such perturbations act as a strong regularizer preventing the network from making highly confident predictions on the training set and thus reducing overfitting. An ensemble strategy is also used for test time augmentation that is shown to further improve the accuracy. The robustness of our CNN models, in terms of reduced generalization error and improved interpretation accuracy is demonstrated with real data examples from the Gulf of Mexico.

scholarly journals Microwave Dielectric Property Retrieval from Open-Ended Coaxial Probe Response with Deep Learning

2021 ◽  
Author(s):  
Cemanur Aydinalp ◽  
Sulayman Joof ◽  
Mehmet Nuri Akinci ◽  
Ibrahim Akduman ◽  
Tuba Yilmaz
Keyword(s):  
Deep Learning ◽  
Dielectric Property ◽  
Large Scale ◽  
Real Data ◽  
Learning Model ◽  
Data Generation ◽  
Retrieval Technique ◽  
Design Flexibility ◽  
A New Technique ◽  
Deep Learning Model

In the manuscript, we propose a new technique for determination of Debye parameters, representing the dielectric properties of materials, from the reflection coefficient response of open-ended coaxial probes. The method retrieves the Debye parameters using a deep learning model designed through utilization of numerically generated data. Unlike real data, using synthetically generated input and output data for training purposes provides representation of a wide variety of materials with rapid data generation. Furthermore, the proposed method provides design flexibility and can be applied to any desired probe with intended dimensions and material. Next, we experimentally verified the designed deep learning model using measured reflection coefficients when the probe was terminated with five different standard liquids, four mixtures,and a gel-like material.and compared the results with the literature. Obtained mean percent relative error was ranging from 1.21±0.06 to 10.89±0.08. Our work also presents a large-scale statistical verification of the proposed dielectric property retrieval technique.

scholarly journals Group Lasso Regularized Deep Learning for Cancer Prognosis from Multi-Omics and Clinical Features

Genes ◽  
2019 ◽  
Vol 10 (3) ◽  
pp. 240 ◽  
Author(s):  
Gangcai Xie ◽  
Chengliang Dong ◽  
Yinfei Kong ◽  
Jiang Zhong ◽  
Mingyao Li ◽  
...  
Keyword(s):  
Deep Learning ◽  
Prior Knowledge ◽  
Clinical Features ◽  
Large Scale ◽  
Real Data ◽  
Group Lasso ◽  
Cancer Prognosis ◽  
Survival Prediction ◽  
Omics Data ◽  
Patients With Cancer

Accurate prognosis of patients with cancer is important for the stratification of patients, the optimization of treatment strategies, and the design of clinical trials. Both clinical features and molecular data can be used for this purpose, for instance, to predict the survival of patients censored at specific time points. Multi-omics data, including genome-wide gene expression, methylation, protein expression, copy number alteration, and somatic mutation data, are becoming increasingly common in cancer studies. To harness the rich information in multi-omics data, we developed GDP (Group lass regularized Deep learning for cancer Prognosis), a computational tool for survival prediction using both clinical and multi-omics data. GDP integrated a deep learning framework and Cox proportional hazard model (CPH) together, and applied group lasso regularization to incorporate gene-level group prior knowledge into the model training process. We evaluated its performance in both simulated and real data from The Cancer Genome Atlas (TCGA) project. In simulated data, our results supported the importance of group prior information in the regularization of the model. Compared to the standard lasso regularization, we showed that group lasso achieved higher prediction accuracy when the group prior knowledge was provided. We also found that GDP performed better than CPH for complex survival data. Furthermore, analysis on real data demonstrated that GDP performed favorably against other methods in several cancers with large-scale omics data sets, such as glioblastoma multiforme, kidney renal clear cell carcinoma, and bladder urothelial carcinoma. In summary, we demonstrated that GDP is a powerful tool for prognosis of patients with cancer, especially when large-scale molecular features are available.

Deep learning-driven velocity model building workflow

2019 ◽  
Vol 38 (11) ◽  
pp. 872a1-872a9 ◽  
Author(s):  
Mauricio Araya-Polo ◽  
Stuart Farris ◽  
Manuel Florez
Keyword(s):  
Deep Learning ◽  
Seismic Data ◽  
Model Building ◽  
Ground Truth ◽  
Velocity Model ◽  
Training Data ◽  
Quality Data ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Velocity Models

Exploration seismic data are heavily manipulated before human interpreters are able to extract meaningful information regarding subsurface structures. This manipulation adds modeling and human biases and is limited by methodological shortcomings. Alternatively, using seismic data directly is becoming possible thanks to deep learning (DL) techniques. A DL-based workflow is introduced that uses analog velocity models and realistic raw seismic waveforms as input and produces subsurface velocity models as output. When insufficient data are used for training, DL algorithms tend to overfit or fail. Gathering large amounts of labeled and standardized seismic data sets is not straightforward. This shortage of quality data is addressed by building a generative adversarial network (GAN) to augment the original training data set, which is then used by DL-driven seismic tomography as input. The DL tomographic operator predicts velocity models with high statistical and structural accuracy after being trained with GAN-generated velocity models. Beyond the field of exploration geophysics, the use of machine learning in earth science is challenged by the lack of labeled data or properly interpreted ground truth, since we seldom know what truly exists beneath the earth's surface. The unsupervised approach (using GANs to generate labeled data)illustrates a way to mitigate this problem and opens geology, geophysics, and planetary sciences to more DL applications.

SaltSeg: Automatic 3D salt segmentation using a deep convolutional neural network

2019 ◽  
Vol 7 (3) ◽  
pp. SE113-SE122 ◽  
Author(s):  
Yunzhi Shi ◽  
Xinming Wu ◽  
Sergey Fomel
Keyword(s):  
Large Scale ◽  
Model Building ◽  
Ground Truth ◽  
Velocity Model ◽  
Training Data ◽  
Data Sets ◽  
Validation Data ◽  
Data Set ◽  
Seismic Image ◽  
Data Generator

Salt boundary interpretation is important for the understanding of salt tectonics and velocity model building for seismic migration. Conventional methods consist of computing salt attributes and extracting salt boundaries. We have formulated the problem as 3D image segmentation and evaluated an efficient approach based on deep convolutional neural networks (CNNs) with an encoder-decoder architecture. To train the model, we design a data generator that extracts randomly positioned subvolumes from large-scale 3D training data set followed by data augmentation, then feed a large number of subvolumes into the network while using salt/nonsalt binary labels generated by thresholding the velocity model as ground truth labels. We test the model on validation data sets and compare the blind test predictions with the ground truth. Our results indicate that our method is capable of automatically capturing subtle salt features from the 3D seismic image with less or no need for manual input. We further test the model on a field example to indicate the generalization of this deep CNN method across different data sets.

scholarly journals A Weakly Supervised and Deep Learning Method for an Additive Topic Analysis of Large Corpora

2019 ◽  
Author(s):  
Yair Fogel-Dror ◽  
Shaul R. Shenhav ◽  
Tamir Sheafer
Keyword(s):  
Content Analysis ◽  
Deep Learning ◽  
Text Classification ◽  
Large Scale ◽  
Low Cost ◽  
Initial Number ◽  
Analysis Method ◽  
Training Set ◽  
Topic Analysis ◽  
Weakly Supervised

The collaborative effort of theory-driven content analysis can benefit significantly from the use of topic analysis methods, which allow researchers to add more categories while developing or testing a theory. This additive approach enables the reuse of previous efforts of analysis or even the merging of separate research projects, thereby making these methods more accessible and increasing the discipline’s ability to create and share content analysis capabilities. This paper proposes a weakly supervised topic analysis method that uses both a low-cost unsupervised method to compile a training set and supervised deep learning as an additive and accurate text classification method. We test the validity of the method, specifically its additivity, by comparing the results of the method after adding 200 categories to an initial number of 450. We show that the suggested method provides a foundation for a low-cost solution for large-scale topic analysis.

scholarly journals DeepFRET: Rapid and automated single molecule FRET data classification using deep learning

2020 ◽  
Author(s):  
Johannes Thomsen ◽  
Magnus B. Sletfjerding ◽  
Stefano Stella ◽  
Bijoya Paul ◽  
Simon Bo Jensen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Structural Biology ◽  
Single Molecule ◽  
Resonance Energy Transfer ◽  
Resonance Energy ◽  
Ground Truth ◽  
Real Data ◽  
Ground Truth Data ◽  
Single Molecule Fret ◽  
Human Operators

AbstractSingle molecule Förster Resonance energy transfer (smFRET) is a mature and adaptable method for studying the structure of biomolecules and integrating their dynamics into structural biology. The development of high throughput methodologies and the growth of commercial instrumentation have outpaced the development of rapid, standardized, and fully automated methodologies to objectively analyze the wealth of produced data. Here we present DeepFRET, an automated standalone solution based on deep learning, where the only crucial human intervention in transiting from raw microscope images to histogram of biomolecule behavior, is a user-adjustable quality threshold. Integrating all standard features of smFRET analysis, DeepFRET will consequently output common kinetic information metrics for biomolecules. We validated the utility of DeepFRET by performing quantitative analysis on simulated, ground truth, data and real smFRET data. The accuracy of classification by DeepFRET outperformed human operators and current commonly used hard threshold and reached >95% precision accuracy only requiring a fraction of the time (<1% as compared to human operators) on ground truth data. Its flawless and rapid operation on real data demonstrates its wide applicability. This level of classification was achieved without any preprocessing or parameter setting by human operators, demonstrating DeepFRET’s capacity to objectively quantify biomolecular dynamics. The provided a standalone executable based on open source code capitalises on the widespread adaptation of machine learning and may contribute to the effort of benchmarking smFRET for structural biology insights.

scholarly journals A Weakly Supervised and Deep Learning Method for an Additive Topic Analysis of Large Corpora

2021 ◽  
Vol 3 (1) ◽  
pp. 29-59
Author(s):  
Yair Fogel-Dror ◽  
Shaul R. Shenhav ◽  
Tamir Sheafer
Keyword(s):  
Content Analysis ◽  
Deep Learning ◽  
Text Classification ◽  
Large Scale ◽  
Low Cost ◽  
Initial Number ◽  
Analysis Method ◽  
Training Set ◽  
Topic Analysis ◽  
Weakly Supervised

Abstract The collaborative effort of theory-driven content analysis can benefit significantly from the use of topic analysis methods, which allow researchers to add more categories while developing or testing a theory. This additive approach enables the reuse of previous efforts of analysis or even the merging of separate research projects, thereby making these methods more accessible and increasing the discipline’s ability to create and share content analysis capabilities. This paper proposes a weakly supervised topic analysis method that uses both a low-cost unsupervised method to compile a training set and supervised deep learning as an additive and accurate text classification method. We test the validity of the method, specifically its additivity, by comparing the results of the method after adding 200 categories to an initial number of 450. We show that the suggested method provides a foundation for a low-cost solution for large-scale topic analysis.

Taking the pulse of the Amazon rainforest by fusing multitemporal Sentinel 1 and 2 data for advanced deep-learning

2021 ◽  
Author(s):  
Melanie Brandmeier ◽  
Eya Cherif
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Land Cover ◽  
Supervised Learning ◽  
Large Scale ◽  
Remote Sensing Data ◽  
Ground Truth ◽  
Brazilian Amazon ◽  
Sensing Data ◽  
Weakly Supervised

&lt;p&gt;Degradation of large forest areas such as the Brazilian Amazon due to logging and fires can increase the human footprint way beyond deforestation. Monitoring and quantifying such changes on a large scale has been addressed by several research groups (e.g. Souza et al. 2013) by making use of freely available remote sensing data such as the Landsat archive. However, fully automatic large-scale land cover/land use mapping is still one of the great challenges in remote sensing. One problem is the availability of reliable &amp;#8220;ground truth&amp;#8221; labels for training supervised learning algorithms. For the Amazon area, several landcover maps with 22 classes are available from the MapBiomas project that were derived by semi-automatic classification and verified by extensive fieldwork (Project MapBiomas). These labels cannot be considered real ground-truth as they were derived from Landsat data themselves but can still be used for weakly supervised training of deep-learning models that have a potential to improve predictions on higher resolution data nowadays available. The term weakly supervised learning was originally coined by (Zhou 2017) and refers to the attempt of constructing predictive models from incomplete, inexact and/or inaccurate labels as is often the case in remote sensing. To this end, we investigate advanced deep-learning strategies on Sentinel-1 timeseries and Sentinel-2 optical data to improve large-scale automatic mapping and monitoring of landcover changes in the Amazon area. Sentinel-1 data has the advantage to be resistant to cloud cover that often hinders optical remote sensing in the tropics.&lt;/p&gt;&lt;p&gt;We propose new architectures that are adapted to the particularities of remote sensing data (S1 timeseries and multispectral S2 data) and compare the performance to state-of-the-art models.&amp;#160; Results using only spectral data were very promising with overall test accuracies of 77.9% for Unet and 74.7% for a DeepLab implementation with ResNet50 backbone and F1 measures of 43.2% and 44.2% respectively.&amp;#160; On the other hand, preliminary results for new architectures leveraging the multi-temporal aspect of &amp;#160;SAR data have improved the quality of mapping, particularly for agricultural classes. For instance, our new designed network AtrousDeepForestM2 has a similar quantitative performances as DeepLab &amp;#160;(F1 of 58.1% vs 62.1%), however it produces better qualitative land cover maps.&lt;/p&gt;&lt;p&gt;To make our approach scalable and feasible for others, we integrate the trained models in a geoprocessing tool in ArcGIS that can also be deployed in a cloud environment and offers a variety of post-processing options to the user.&lt;/p&gt;&lt;p&gt;Souza, J., Carlos M., et al. (2013). &quot;Ten-Year Landsat Classification of Deforestation and Forest Degradation in the Brazilian Amazon.&quot; Remote Sensing 5(11): 5493-5513.&amp;#160;&amp;#160;&amp;#160;&lt;/p&gt;&lt;p&gt;Zhou, Z.-H. (2017). &quot;A brief introduction to weakly supervised learning.&quot; National Science Review 5(1): 44-53.&lt;/p&gt;&lt;p&gt;&quot;Project MapBiomas - Collection&amp;#160; 4.1 of Brazilian Land Cover &amp; Use Map Series, accessed on January 2020 through the link: https://mapbiomas.org/colecoes-mapbiomas?cama_set_language=en&quot;&lt;/p&gt;

scholarly journals Predicting ocean-induced ice-shelf melt rates using a machine learning image segmentation approach

2022 ◽  
Author(s):  
Sebastian Harry Reid Rosier ◽  
Christopher Bull ◽  
G. Hilmar Gudmundsson
Keyword(s):  
Deep Learning ◽  
Ocean Circulation ◽  
Large Scale ◽  
Ice Sheet ◽  
Ground Truth ◽  
Ocean Model ◽  
Ice Shelf ◽  
Computational Expense ◽  
Shelf Slope ◽  
Wide Range

Abstract. Through their role in buttressing upstream ice flow, Antarctic ice shelves play an important part in regulating future sea level change. Reduction in ice-shelf buttressing caused by increased ocean-induced melt along their undersides is now understood to be one of the key drivers of ice loss from the Antarctic Ice Sheet. However, despite the importance of this forcing mechanism most ice-sheet simulations currently rely on simple melt-parametrisations of this ocean-driven process, since a fully coupled ice-ocean modelling framework is prohibitively computationally expensive. Here, we provide an alternative approach that is able to capture the greatly improved physical description of this process provided by large-scale ocean-circulation models over currently employed melt-parameterisations but with trivial computational expense. We introduce a new approach that brings together deep learning and physical modelling to develop a deep neural network framework, MELTNET, that can emulate ocean model predictions of sub-ice shelf melt rates. We train MELTNET on synthetic geometries, using the NEMO ocean model as a ground-truth in lieu of observations to provide melt rates both for training and to evaluate the performance of the trained network. We show that MELTNET can accurately predict melt rates for a wide range of complex synthetic geometries and outperforms more traditional parameterisations for > 95 % of geometries tested. Furthermore, we find MELTNET's melt rate estimates show sensitivity to established physical relationships such as a changes in thermal forcing and ice shelf slope. This study demonstrates the potential for a deep learning framework to calculate melt rates with almost no computational expense, that could in the future be used in conjunction with an ice sheet model to provide predictions for large-scale ice sheet models.

scholarly journals Two hours in Hollywood: A manually annotated ground truth data set of eye movements during movie clip watching

2020 ◽  
Vol 13 (4) ◽  
Author(s):  
Ioannis Agtzidis ◽  
Mikhail Startsev ◽  
Michael Dorr
Keyword(s):  
Eye Movement ◽  
Large Scale ◽  
Ground Truth ◽  
Training Set ◽  
Event Type ◽  
Short Article ◽  
Data Set ◽  
Ground Truth Data ◽  
The Gaze ◽  
Movie Clip

In this short article we present our manual annotation of the eye movement events in a subset of the large-scale eye tracking data set Hollywood2. Our labels include fixations, saccades, and smooth pursuits, as well as a noise event type (the latter representing either blinks, loss of tracking, or physically implausible signals). In order to achieve more consistent annotations, the gaze samples were labelled by a novice rater based on rudimentary algorithmic suggestions, and subsequently corrected by an expert rater. Overall, we annotated eye movement events in the recordings corresponding to 50 randomly selected test set clips and 6 training set clips from Hollywood2, which were viewed by 16 observers and amount to a total of approximately 130 minutes of gaze data. In these labels, 62.4% of the samples were attributed to fixations, 9.1% – to saccades, and, notably, 24.2% – to pursuit (the remainder marked as noise). After evaluation of 15 published eye movement classification algorithms on our newly collected annotated data set, we found that the most recent algorithms perform very well on average, and even reach human-level labelling quality for fixations and saccades, but all have a much larger room for improvement when it comes to smooth pursuit classification. The data set is made available at https://gin.g- node.org/ioannis.agtzidis/hollywood2_em.

Sign in / Sign up

Export Citation Format

Share Document