Semi-Supervised Salt Segmentation Using Mean Teacher

2022 ◽  
pp. 1-39
Author(s):  
Zhicheng Geng ◽  
Zhanxuan Hu ◽  
Xinming Wu ◽  
Luming Liang ◽  
Sergey Fomel
Keyword(s):  
Network Architecture ◽  
Input Data ◽  
Moving Average ◽  
Training Data ◽  
Segmentation Method ◽  
Student Model ◽  
The Mean ◽  
Seismic Images ◽  
Subsurface Modeling ◽  
Extract Information

Detecting subsurface salt structures from seismic images is important for seismic structural analysis and subsurface modeling. Recently, deep learning has been successfully applied in solving salt segmentation problems. However, most of the studies focus on supervised salt segmentation and require numerous accurately labeled data, which is usually laborious and time-consuming to collect, especially for the geophysics community. In this paper, we propose a semi-supervised framework for salt segmentation, which requires only a small amount of labeled data. In our method, adopting the mean teacher method, we train two models sharing the same network architecture. The student model is optimized using a combination of supervised loss and unsupervised consistency loss, whereas the teacher model is the exponential moving average (EMA) of the student model. We introduce the unsupervised consistency loss to better extract information from unlabeled data by constraining the network to give consistent predictions for the input data and its perturbed version. We train and test our novel semi-supervised method on both synthetic and real datasets. Results demonstrate that our proposed semi-supervised salt segmentation method outperforms the supervised baseline when there is a lack of labeled training data.

scholarly journals Robust Approach to Supervised Deep Neural Network Training for Real-Time Object Classification in Cluttered Indoor Environment

2021 ◽  
Vol 11 (15) ◽  
pp. 7148
Author(s):  
Bedada Endale ◽  
Abera Tullu ◽  
Hayoung Shi ◽  
Beom-Soo Kang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real Time ◽  
Network Architecture ◽  
Input Data ◽  
Deep Neural Network ◽  
Data Augmentation ◽  
Object Classification ◽  
Training Data ◽  
Gradient Descent Algorithm

Unmanned aerial vehicles (UAVs) are being widely utilized for various missions: in both civilian and military sectors. Many of these missions demand UAVs to acquire artificial intelligence about the environments they are navigating in. This perception can be realized by training a computing machine to classify objects in the environment. One of the well known machine training approaches is supervised deep learning, which enables a machine to classify objects. However, supervised deep learning comes with huge sacrifice in terms of time and computational resources. Collecting big input data, pre-training processes, such as labeling training data, and the need for a high performance computer for training are some of the challenges that supervised deep learning poses. To address these setbacks, this study proposes mission specific input data augmentation techniques and the design of light-weight deep neural network architecture that is capable of real-time object classification. Semi-direct visual odometry (SVO) data of augmented images are used to train the network for object classification. Ten classes of 10,000 different images in each class were used as input data where 80% were for training the network and the remaining 20% were used for network validation. For the optimization of the designed deep neural network, a sequential gradient descent algorithm was implemented. This algorithm has the advantage of handling redundancy in the data more efficiently than other algorithms.

Semiautomatic first-arrival picking of microseismic events by using the pixel-wise convolutional image segmentation method

Geophysics ◽  
2019 ◽  
Vol 84 (3) ◽  
pp. V143-V155 ◽  
Author(s):  
Hao Wu ◽  
Bo Zhang ◽  
Fangyu Li ◽  
Naihao Liu
Keyword(s):  
Image Segmentation ◽  
Network Architecture ◽  
Training Data ◽  
Segmentation Method ◽  
Fracture Detection ◽  
Time Index ◽  
Network Hierarchy ◽  
First Arrival ◽  
Microseismic Events ◽  
Trained Network

Microseismic imaging plays an important role in hydraulic fracture detection, and the first-arrival picking of microseismic events is the bedrock of microseismic imaging. Manual picking is the most reliable and also the most time-consuming method for the detection of the first arrival of microseismic events. Accurate and efficient first-arrival picking in a real noisy environment is a challenge for most of the automatic first-arrival picking methods. We have developed a novel workflow to automatically pick the first arrival of microseismics by using a state-of-the art pixel-wise convolutional image segmentation method. We first form the training data by randomly selecting part of the microseismic traces and manually pick the time index of the first arrivals. Next, we segment the selected traces into two parts according to the time index of manual picking and assign each part a label accordingly. Then, we build an encoder-decoder convolutional neural network architecture and use the training data and training label as the input. Next, we obtain the trained network hierarchy by learning the segmented training data and labels. Finally, we predict the first arrivals of microseismic events by applying the trained network hierarchy to the rest of the microseismic traces. The synthetic and field data examples demonstrate that our method successfully identifies the first arrivals. The predicted first-arrival result obtained by using our method is superior to the result obtained by using the traditional method of short-term average and long-term average.

AN EXPONENTIALLY WEIGHTED MOVING AVERAGE METHOD WITH DESIGNED INPUT DATA ASSIGNMENTS FOR FORECASTING LIME PRICES IN THAILAND

2017 ◽  
Vol 79 (6) ◽  
Author(s):  
Thitima Booranawong ◽  
Apidet Booranawong
Keyword(s):  
Input Data ◽  
Moving Average ◽  
Weighting Factor ◽  
Percentage Error ◽  
Exponentially Weighted Moving Average ◽  
Monthly Data ◽  
Forecasting Error ◽  
The Mean ◽  
Weighted Moving Average ◽  
Exponentially Weighted

In this paper, the Exponentially Weighted Moving Average (EWMA) method with designed input data assignments (i.e. the proposed method) is presented to forecast lime prices in Thailand during January 2016 to December 2016. The lime prices from January 2011 to December 2015 as the input data are gathered from the website’s database of Simummuang market, which is one of the big markets in Thailand. The novelty of our paper is that although the performance of the EWMA method significantly decreases when applying to forecast data which show trend and seasonality behaviors and the EWMA method is used for short-range forecasting (i.e. usually one month into the future), the proposed method can properly handle such mentioned problems. For this purpose, to forecast lime prices, five different input data are intently defined before assigned to the EWMA method: a) the monthly data of the year 2015 (i.e. the recent year data), b) the average monthly data of the year 2011 to 2015, c) the median of the monthly data of the year 2011 to 2015, d) the monthly data of the year 2011 to 2015 after applying the linear weighting factor, where the higher weight value is applied to the recent data, and e) the average monthly data of the year 2011 to 2015 after applying the exponential weighting factor, where the higher weight is also applied to the recent data. These designed input data are used as agents of the raw data. Our study reveals that using the input data b) with the EWMA method to forecast lime prices during January 2016 to September 2016 gives the smallest forecasting error measured by the Mean Absolute Percentage Error (MAPE). Forecasted lime prices of October 2016 to December 2016 are also provided. Additionally, we demonstrate that the proposed method works well compared with the Double Exponentially Weighted Moving Average (DEWMA), the Multiplicative Holt-Winters (MHW), and the Additive Holt-Winters (AHW) methods, which are suitably used for forecasting data with the trend and the seasonality.

scholarly journals Data-Driven Stability Assessment of Multilayer Long Short-Term Memory Networks

2021 ◽  
Vol 11 (4) ◽  
pp. 1829
Author(s):  
Davide Grande ◽  
Catherine A. Harris ◽  
Giles Thomas ◽  
Enrico Anderlini
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Short Term Memory ◽  
Moving Average ◽  
Machine Learning Algorithms ◽  
Physical Models ◽  
Data Driven ◽  
The Stability ◽  
And Control ◽  
Nonlinear Autoregressive

Recurrent Neural Networks (RNNs) are increasingly being used for model identification, forecasting and control. When identifying physical models with unknown mathematical knowledge of the system, Nonlinear AutoRegressive models with eXogenous inputs (NARX) or Nonlinear AutoRegressive Moving-Average models with eXogenous inputs (NARMAX) methods are typically used. In the context of data-driven control, machine learning algorithms are proven to have comparable performances to advanced control techniques, but lack the properties of the traditional stability theory. This paper illustrates a method to prove a posteriori the stability of a generic neural network, showing its application to the state-of-the-art RNN architecture. The presented method relies on identifying the poles associated with the network designed starting from the input/output data. Providing a framework to guarantee the stability of any neural network architecture combined with the generalisability properties and applicability to different fields can significantly broaden their use in dynamic systems modelling and control.

scholarly journals Real-Time Instance Segmentation of Traffic Videos for Embedded Devices

Sensors ◽  
2021 ◽  
Vol 21 (1) ◽  
pp. 275
Author(s):  
Ruben Panero Martinez ◽  
Ionut Schiopu ◽  
Bruno Cornelis ◽  
Adrian Munteanu
Keyword(s):  
Real Time ◽  
Network Architecture ◽  
Training Procedure ◽  
Segmentation Method ◽  
Embedded Devices ◽  
Network Training ◽  
Assignment Algorithm ◽  
Ablation Study ◽  
Reduced Rate ◽  
Instance Segmentation

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.

scholarly journals Reservoir Sizing at Draft Level of 75% of Mean Annual Flow Using Drought Magnitude Based Method on Canadian Rivers

Hydrology ◽  
2021 ◽  
Vol 8 (2) ◽  
pp. 79
Author(s):  
Tribeni C. Sharma ◽  
Umed S. Panu
Keyword(s):  
Standard Deviation ◽  
Time Scale ◽  
Moving Average ◽  
Flow Data ◽  
Truncation Level ◽  
Drought Intensity ◽  
Error Margin ◽  
Reservoir Volume ◽  
The Mean ◽  
Reservoir Sizing

On a global basis, there is trend that a majority of reservoirs are sized using a draft of 75% of the mean annual flow (0.75 MAF). The reservoir volumes based on the proposed drought magnitude (DM) method and the sequent peak algorithm (SPA) at 0.75 MAF draft were compared at the annual, monthly and weekly scales using the flow sequences of 25 Canadian rivers. In our assessment, the monthly scale is adequate for such analyses. The DM method, although capable of using flow data at any time scale, has been demonstrated using monthly standardized hydrological index (SHI) sequences. The moving average (MA) smoothing of the monthly SHI sequences formed the basis in the DM method for estimating the reservoir volume through the use of the extreme number theorem, and the hypothesis that drought magnitude is equal to the product of the drought intensity and drought length. The truncation level in the SHI sequences was found as SHIo [ = (0.75 ‒ 1) µo/σo], where µo and σo are the overall mean and standard deviation of the monthly flows. The DM-based estimates for the deficit volumes and the SPA-based reservoir volumes were found comparable within an error margin of ±18%.

scholarly journals DeepIsoFun: a deep domain adaptation approach to predict isoform functions

2018 ◽  
Vol 35 (15) ◽  
pp. 2535-2544 ◽  
Author(s):  
Dipan Shaw ◽  
Hao Chen ◽  
Tao Jiang
Keyword(s):  
Network Architecture ◽  
Domain Adaptation ◽  
Multiple Instance Learning ◽  
Training Data ◽  
Supplementary Information ◽  
Operating Characteristics ◽  
Neural Network Architecture ◽  
Receiver Operating Characteristics Curve ◽  
Gene Functions ◽  
Better Than

AbstractMotivationIsoforms are mRNAs produced from the same gene locus by alternative splicing and may have different functions. Although gene functions have been studied extensively, little is known about the specific functions of isoforms. Recently, some computational approaches based on multiple instance learning have been proposed to predict isoform functions from annotated gene functions and expression data, but their performance is far from being desirable primarily due to the lack of labeled training data. To improve the performance on this problem, we propose a novel deep learning method, DeepIsoFun, that combines multiple instance learning with domain adaptation. The latter technique helps to transfer the knowledge of gene functions to the prediction of isoform functions and provides additional labeled training data. Our model is trained on a deep neural network architecture so that it can adapt to different expression distributions associated with different gene ontology terms.ResultsWe evaluated the performance of DeepIsoFun on three expression datasets of human and mouse collected from SRA studies at different times. On each dataset, DeepIsoFun performed significantly better than the existing methods. In terms of area under the receiver operating characteristics curve, our method acquired at least 26% improvement and in terms of area under the precision-recall curve, it acquired at least 10% improvement over the state-of-the-art methods. In addition, we also study the divergence of the functions predicted by our method for isoforms from the same gene and the overall correlation between expression similarity and the similarity of predicted functions.Availability and implementationhttps://github.com/dls03/DeepIsoFun/Supplementary informationSupplementary data are available at Bioinformatics online.

Inoptimality losses in forest management decisions caused by errors in an inventory based on airborne laser scanning and aerial photographs

2010 ◽  
Vol 40 (12) ◽  
pp. 2427-2438 ◽  
Author(s):  
Md. Nurul Islam ◽  
Mikko Kurttila ◽  
Lauri Mehtätalo ◽  
Timo Pukkala
Keyword(s):  
Input Data ◽  
Laser Scanning ◽  
Basal Area ◽  
Airborne Laser Scanning ◽  
Aerial Photographs ◽  
Stand Age ◽  
Inventory Data ◽  
Airborne Laser ◽  
Mean Diameter ◽  
The Mean

Errors in inventory data may lead to inoptimal decisions that ultimately result in financial losses for forest owners. We estimated the expected monetary losses resulting from data errors that are similar to errors in laser-based forest inventory. The mean loss was estimated for 67 stands by simulating 100 realizations of inventory data for each stand with errors that mimic those in airborne laser scanning (ALS) based inventory. These realizations were used as input data in stand management optimization, which maximized the present value of all future net incomes (NPV). The inoptimality loss was calculated as the difference between the NPV of the optimal solution and the true NPV of the solution obtained with erroneous input data. The results showed that the mean loss exceeded €300·ha–1 (US$425·ha–1) in 84% of the stands. On average, the losses increased with decreasing stand age and mean diameter. Furthermore, increasing errors in the basal area weighted mean diameter and basal area of spruce were found to significantly increase the loss. It has been discussed that improvements in the accuracy of ALS-based inventory could be financially justified.

scholarly journals Deep learning of stochastic contagion dynamics on complex networks

2020 ◽  
Author(s):  
Charles Murphy ◽  
Edward Laurence ◽  
Antoine Allard
Keyword(s):  
Deep Learning ◽  
Network Architecture ◽  
Time Series Data ◽  
Training Data ◽  
Series Data ◽  
Neural Network Architecture ◽  
Complementary Approach ◽  
Quantitative Accuracy ◽  
Simplifying Assumptions ◽  
Effective Models

Abstract Forecasting the evolution of contagion dynamics is still an open problem to which mechanistic models only offer a partial answer. To remain mathematically and/or computationally tractable, these models must rely on simplifying assumptions, thereby limiting the quantitative accuracy of their predictions and the complexity of the dynamics they can model. Here, we propose a complementary approach based on deep learning where the effective local mechanisms governing a dynamic are learned automatically from time series data. Our graph neural network architecture makes very few assumptions about the dynamics, and we demonstrate its accuracy using stochastic contagion dynamics of increasing complexity on static and temporal networks. By allowing simulations on arbitrary network structures, our approach makes it possible to explore the properties of the learned dynamics beyond the training data. Our results demonstrate how deep learning offers a new and complementary perspective to build effective models of contagion dynamics on networks.

scholarly journals Action detection using a neural network elucidates the genetics of mouse grooming behavior

2020 ◽  
Author(s):  
Brian Q. Geuther ◽  
Asaf Peer ◽  
Hao He ◽  
Gautam Sabnis ◽  
Vivek M. Philip ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Stereotyped Behavior ◽  
General Purpose ◽  
Training Data ◽  
Mouse Strains ◽  
Human Observer ◽  
Grooming Behavior ◽  
Action Detection ◽  
Visual Diversity

AbstractAutomated detection of complex animal behaviors remains a challenging problem in neuroscience, particularly for behaviors that consist of disparate sequential motions. Grooming, a prototypical stereotyped behavior, is often used as an endophenotype in psychiatric genetics. Using mouse grooming behavior as an example, we develop a general purpose neural network architecture capable of dynamic action detection at human observer-level performance and operate across dozens of mouse strains with high visual diversity. We provide insights into the amount of human annotated training data that are needed to achieve such performance. We survey grooming behavior in the open field in 2500 mice across 62 strains, determine its heritable components, conduct GWAS to outline its genetic architecture, and perform PheWAS to link human psychiatric traits through shared underlying genetics. Our general machine learning solution that automatically classifies complex behaviors in large datasets will facilitate systematic studies of mechanisms underlying these behaviors.

Sign in / Sign up

Export Citation Format

Share Document