scholarly journals Nanopore callers for epigenetics from limited supervised data

2021 ◽  
Author(s):  
Brian Yao ◽  
Chloe Hsu ◽  
Gal Goldner ◽  
Yael Michaeli ◽  
Yuval Ebenstein ◽  
...  
Keyword(s):  
Incomplete Data ◽  
Deep Neural Networks ◽  
Markov Models ◽  
Hidden Markov ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Training Dataset ◽  
Nanopore Sequencing ◽  
Base Modifications ◽  
Sequencing Platforms

Nanopore sequencing platforms combined with supervised machine learning (ML) have been effective at detecting base modifications in DNA such as 5mC and 6mA. These ML-based nanopore callers have typically been trained on data that span all modifications on all possible DNA k-mer backgrounds—a complete training dataset. However, as nanopore technology is pushed to more and more epigenetic modifications, such complete training data will not be feasible to obtain. Nanopore calling has historically been performed with Hidden Markov Models (HMMs) that cannot make successful calls for k-mer contexts not seen during training because of their independent emission distributions. However, deep neural networks (DNNs), which share parameters across contexts, are increasingly being used as callers, often outperforming their HMM cousins. It stands to reason that a DNN approach should be able to better generalize to unseen k-mer contexts. Indeed, herein we demonstrate that a common DNN approach (DeepSignal) outperforms a common HMM approach (Nanopolish) in the incomplete data setting. Furthermore, we propose a novel hybrid HMM-DNN approach, Amortized-HMM, that outperforms both the pure HMM and DNN approaches on 5mC calling when the training data are incomplete. Such an approach is expected to be useful for calling 5hmC and combinations of cytosine modifications, where complete training data are not likely to be available.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

2017 ◽  
Vol 2017 ◽  
pp. 1-10 ◽  
Author(s):  
Cédric Beaulac ◽  
Fabrice Larribe
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Hidden Markov Models ◽  
Real Time ◽  
Markov Models ◽  
Hidden Markov ◽  
Time Estimation ◽  
Supervised Machine Learning ◽  
Real Time Estimation ◽  
Mobile Target

We propose to use a supervised machine learning technique to track the location of a mobile agent in real time. Hidden Markov Models are used to build artificial intelligence that estimates the unknown position of a mobile target moving in a defined environment. This narrow artificial intelligence performs two distinct tasks. First, it provides real-time estimation of the mobile agent’s position using the forward algorithm. Second, it uses the Baum–Welch algorithm as a statistical learning tool to gain knowledge of the mobile target. Finally, an experimental environment is proposed, namely, a video game that we use to test our artificial intelligence. We present statistical and graphical results to illustrate the efficiency of our method.

scholarly journals Development of Vietnamese Speech Synthesis System using Deep Neural Networks

2019 ◽  
Vol 34 (4) ◽  
pp. 349-363 ◽  
Author(s):  
Thinh Van Nguyen ◽  
Bao Quoc Nguyen ◽  
Kinh Huy Phan ◽  
Hai Van Do
Keyword(s):  
Neural Networks ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Speech Synthesis ◽  
Deep Neural Networks ◽  
Hidden Markov ◽  
Experimental Results ◽  
Training Data ◽  
The Internet ◽  
Synthesis System

In this paper, we present our first Vietnamese speech synthesis system based on deep neural networks. To improve the training data collected from the Internet, a cleaning method is proposed. The experimental results indicate that by using deeper architectures we can achieve better performance for the TTS than using shallow architectures such as hidden Markov model. We also present the effect of using different amounts of data to train the TTS systems. In the VLSP TTS challenge 2018, our proposed DNN-based speech synthesis system won the first place in all three subjects including naturalness, intelligibility, and MOS.

scholarly journals Learning with small amount of data on various task

2020 ◽  
Author(s):  
Stefanie
Keyword(s):  
Deep Neural Networks ◽  
Learning Strategy ◽  
Teaching Strategy ◽  
Training Sample ◽  
Learning To Teach ◽  
Training Data ◽  
Training Dataset ◽  
Small Data ◽  
Training Samples ◽  
Teacher Model

As a student, I am learning knowledge with the help of teachers and the teacher plays a crucial role in our life. A wonderful instructor is able to teach a student with appropriate teaching materials. Therefore, in this project, I explore a teaching strategy called learning to teach (L2T) in which a teacher model could provide high-quality training samples to a student model. However, one major problem of L2T is that the teacher model will only select a subset of the training dataset as the final training data for the student. Learning to teach small-data learning strategy (L2TSDL) is proposed to solve this problem. In this strategy, the teacher model will calculate the importance score for every training sample and help students to make use of all training samples. To demonstrate the advantage of the proposed approach over L2T, I take the training of different deep neural networks (DNN) on image classification task as an exampleand show that L2TSDL could achieve good performance on both large and small dataset.

scholarly journals Improving peptide-spectrum matching by fragmentation prediction using Hidden Markov Models

2018 ◽  
Author(s):  
Ufuk Kirik ◽  
Jan C. Refsgaard ◽  
Lars J. Jensen
Keyword(s):  
Charge State ◽  
Markov Models ◽  
Hidden Markov ◽  
Peptide Identification ◽  
Training Data ◽  
Prediction Algorithm ◽  
Candidate Peptide ◽  
Fragment Ions ◽  
Robust Model ◽  
Spectrum Matching

AbstractTandem mass-spectrometry has become the method of choice for high-throughput, quantitative analysis in proteomics. However, since the link between the peptides and the proteins they originate from is typically broken, identification of the analyzed peptides relies on matching of the fragmentation spectra (MS2) to theoretical spectra of possible candidate peptides, often filtered for precursor ion mass. To this end, peptide-spectrum matching algorithms score the concordance between the experimental and the theoretical spectra of candidate peptides, by evaluating the number (or proportion) of theoretically possible fragment ions observed in the experimental spectra, without any discrimination. However, the assumption that each theoretical fragment is just as likely to be observed is inaccurate. On the contrary, MS2 spectra often have few dominant fragments.We propose a novel prediction algorithm based on a hidden Markov model, which allow for the training process to be carried out very efficiently. Using millions of MS/MS spectra generated in our lab, we found an overall good reproducibility across different fragmentation spectra, given the precursor peptide and charge state. This result implies that there is indeed a pattern to fragmentation that justifies using machine learning methods. Furthermore, the overall agreement between spectra of the same peptide at the same charge state serves as an upper limit on how well prediction algorithms can be expected to perform.We have investigated the performance of a third order HMM model, trained on several million MS2 spectra, in various ways. Compared to a mock model, in which the fragment ions and their intensities are shuffled, we see a clear difference in prediction accuracy using our model. This result indicates that our model can pick up meaningful patterns, i.e. we can indeed learn the fragmentation process. Secondly, looking at the variability of the prediction performance by varying the train/test data split, in a K-fold cross validation scheme, we observed an overall robust model that performs well independent of the specific peptides that are present in the training data.Last but not least, we propose that the real value of this model is as a pre-processing step in the peptide identification process, by discerning fragment ions that are unlikely to be intense for a given candidate peptide, rather than using the actual predicted intensities. As such, probabilistic measures of concordance between experimental and theoretical spectra, would leverage better statistics.

Semi-supervised learning of Hidden Markov Models for biological sequence analysis

2018 ◽  
Vol 35 (13) ◽  
pp. 2208-2215 ◽  
Author(s):  
Ioannis A Tamposis ◽  
Konstantinos D Tsirigos ◽  
Margarita C Theodoropoulou ◽  
Panagiota I Kontou ◽  
Pantelis G Bagos
Keyword(s):  
Sequence Analysis ◽  
Supervised Learning ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Transmembrane Protein ◽  
Training Data ◽  
Supplementary Information ◽  
Training Procedure ◽  
Partially Labeled Data

Abstract Motivation Hidden Markov Models (HMMs) are probabilistic models widely used in applications in computational sequence analysis. HMMs are basically unsupervised models. However, in the most important applications, they are trained in a supervised manner. Training examples accompanied by labels corresponding to different classes are given as input and the set of parameters that maximize the joint probability of sequences and labels is estimated. A main problem with this approach is that, in the majority of the cases, labels are hard to find and thus the amount of training data is limited. On the other hand, there are plenty of unclassified (unlabeled) sequences deposited in the public databases that could potentially contribute to the training procedure. This approach is called semi-supervised learning and could be very helpful in many applications. Results We propose here, a method for semi-supervised learning of HMMs that can incorporate labeled, unlabeled and partially labeled data in a straightforward manner. The algorithm is based on a variant of the Expectation-Maximization (EM) algorithm, where the missing labels of the unlabeled or partially labeled data are considered as the missing data. We apply the algorithm to several biological problems, namely, for the prediction of transmembrane protein topology for alpha-helical and beta-barrel membrane proteins and for the prediction of archaeal signal peptides. The results are very promising, since the algorithms presented here can significantly improve the prediction performance of even the top-scoring classifiers. Supplementary information Supplementary data are available at Bioinformatics online.

Observable Operator Models for Discrete Stochastic Time Series

2000 ◽  
Vol 12 (6) ◽  
pp. 1371-1398 ◽  
Author(s):  
Herbert Jaeger
Keyword(s):  
Hidden Markov Models ◽  
Stochastic Systems ◽  
Markov Models ◽  
Learning Algorithm ◽  
Hidden Markov ◽  
Training Data ◽  
Constructive Learning ◽  
Proper Subclass ◽  
Stochastic Time Series ◽  
Dependent Processes

A widely used class of models for stochastic systems is hidden Markov models. Systems that can be modeled by hidden Markov models are a proper subclass of linearly dependent processes, a class of stochastic systems known from mathematical investigations carried out over the past four decades. This article provides a novel, simple characterization of linearly dependent processes, called observable operator models. The mathematical properties of observable operator models lead to a constructive learning algorithm for the identification of linearly dependent processes. The core of the algorithm has a time complexity of O (N + nm3), where N is the size of training data, n is the number of distinguishable outcomes of observations, and m is model state-space dimension.

scholarly journals Probability Model Based on Cluster Analysis to Classify Sequences of Observations for Small Training Sets

2020 ◽  
Vol 8 (1) ◽  
pp. 296-303
Author(s):  
Sergey S Yulin ◽  
Irina N Palamar
Keyword(s):  
Markov Chain ◽  
Markov Model ◽  
Hidden Markov Model ◽  
Short Term Memory ◽  
Hidden Markov ◽  
Probability Model ◽  
Training Data ◽  
Training Dataset ◽  
Experimental Comparison ◽  
Self Organizing Map

The problem of recognizing patterns, when there are few training data available, is particularly relevant and arises in cases when collection of training data is expensive or essentially impossible. The work proposes a new probability model MC&CL (Markov Chain and Clusters) based on a combination of markov chain and algorithm of clustering (self-organizing map of Kohonen, k-means method), to solve a problem of classifying sequences of observations, when the amount of training dataset is low. An original experimental comparison is made between the developed model (MC&CL) and a number of the other popular models to classify sequences: HMM (Hidden Markov Model), HCRF (Hidden Conditional Random Fields),LSTM (Long Short-Term Memory), kNN+DTW (k-Nearest Neighbors algorithm + Dynamic Time Warping algorithm). A comparison is made using synthetic random sequences, generated from the hidden markov model, with noise added to training specimens. The best accuracy of classifying the suggested model is shown, as compared to those under review, when the amount of training data is low.

Sign in / Sign up

Export Citation Format

Share Document