scholarly journals Improving Convolutional Network Interpretability with Exponential Activations

2019 ◽  
Author(s):  
Peter K. Koo ◽  
Matt Ploenzke
Keyword(s):  
Dna Sequences ◽  
Ground Truth ◽  
Genomic Sequences ◽  
Sequence Motifs ◽  
Convolutional Network ◽  
Distributed Representations ◽  
Convolutional Networks ◽  
Synthetic Dna

AbstractDeep convolutional networks trained on regulatory genomic sequences tend to learn distributed representations of sequence motifs across many first layer filters. This makes it challenging to decipher which features are biologically meaningful. Here we introduce the exponential activation that – when applied to first layer filters – leads to more interpretable representations of motifs, both visually and quantitatively, compared to rectified linear units. We demonstrate this on synthetic DNA sequences which have ground truth with various convolutional networks, and then show that this phenomenon holds on in vivo DNA sequences.

scholarly journals Improving representations of genomic sequence motifs in convolutional networks with exponential activations

2020 ◽  
Author(s):  
Peter K. Koo ◽  
Matt Ploenzke
Keyword(s):  
Neural Networks ◽  
Dna Sequences ◽  
Test Performance ◽  
Genomic Sequence ◽  
Comprehensive Analysis ◽  
Sequence Motifs ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Learned Features

ABSTRACTDeep convolutional neural networks (CNNs) trained on regulatory genomic sequences tend to build representations in a distributed manner, making it a challenge to extract learned features that are biologically meaningful, such as sequence motifs. Here we perform a comprehensive analysis on synthetic sequences to investigate the role that CNN activations have on model interpretability. We show that employing an exponential activation to first layer filters consistently leads to interpretable and robust representations of motifs compared to other commonly used activations. Strikingly, we demonstrate that CNNs with better test performance do not necessarily imply more interpretable representations with attribution methods. We find that CNNs with exponential activations significantly improve the efficacy of recovering biologically meaningful representations with attribution methods. We demonstrate these results generalise to real DNA sequences across several in vivo datasets. Together, this work demonstrates how a small modification to existing CNNs, i.e. setting exponential activations in the first layer, can significantly improve the robustness and interpretabilty of learned representations directly in convolutional filters and indirectly with attribution methods.

scholarly journals Nucleosome positioning on large tandem DNA repeats of the '601' sequence engineered in Saccharomyces cerevisiae

2021 ◽  
Author(s):  
Astrid Lancrey ◽  
Alexandra Joubert ◽  
Evelyne Duvernois-Berthet ◽  
Etienne Routhier ◽  
Saurabh Raj ◽  
...  
Keyword(s):  
Saccharomyces Cerevisiae ◽  
Dna Sequence ◽  
Dna Sequences ◽  
Nucleosome Occupancy ◽  
Nucleosome Positioning ◽  
Dna Repeats ◽  
Repeat Array ◽  
Synthetic Dna

The so-called 601 DNA sequence is often used to constrain the position of nucleosomes on a DNA molecule in vitro. Although the ability of the 147 base pair sequence to precisely position a nucleosome in vitro is well documented, in vivo application of this property has been explored only in a few studies and yielded contradictory conclusions. Our goal in the present study was to test the ability of the 601 sequence to dictate nucleosome positioning in Saccharomyces cerevisiae in the context of a long tandem repeat array inserted in a yeast chromosome. We engineered such arrays with three different repeat size, namely 167, 197 and 237 base pairs. Although our arrays are able to position nucleosomes in vitro as expected, analysis of nucleosome occupancy on these arrays in vivo revealed that nucleosomes are not preferentially positioned as expected on the 601-core sequence along the repeats and that the measured nucleosome repeat length does not correspond to the one expected by design. Altogether our results demonstrate that the rules defining nucleosome positions on this DNA sequence in vitro are not valid in vivo, at least in this chromosomal context, questioning the relevance of using the 601 sequence in vivo to achieve precise nucleosome positioning on designer synthetic DNA sequences.

scholarly journals DeeperBind: Enhancing Prediction of Sequence Specificities of DNA Binding Proteins

2017 ◽  
Author(s):  
Hamid Reza Hassanzadeh ◽  
May D. Wang
Keyword(s):  
Dna Binding ◽  
Protein Binding ◽  
High Throughput ◽  
Dna Sequences ◽  
Dynamics Modeling ◽  
Convolutional Network ◽  
Dna Promoters ◽  
Downstream Analysis

AbstractTranscription factors (TFs) are macromolecules that bind to cis-regulatory specific sub-regions of DNA promoters and initiate transcription. Finding the exact location of these binding sites (aka motifs) is important in a variety of domains such as drug design and development. To address this need, several in vivo and in vitro techniques have been developed so far that try to characterize and predict the binding specificity of a protein to different DNA loci. The major problem with these techniques is that they are not accurate enough in prediction of the binding affinity and characterization of the corresponding motifs. As a result, downstream analysis is required to uncover the locations where proteins of interest bind. Here, we propose DeeperBind, a long short term recurrent convolutional network for prediction of protein binding specificities with respect to DNA probes. DeeperBind can model the positional dynamics of probe sequences and hence reckons with the contributions made by individual sub-regions in DNA sequences, in an effective way. Moreover, it can be trained and tested on datasets containing varying-length sequences. We apply our pipeline to the datasets derived from protein binding microarrays (PBMs), an in-vitro high-throughput technology for quantification of protein-DNA binding preferences, and present promising results. To the best of our knowledge, this is the most accurate pipeline that can predict binding specificities of DNA sequences from the data produced by high-throughput technologies through utilization of the power of deep learning for feature generation and positional dynamics modeling.

scholarly journals A Calibrated Multiexit Neural Network for Detecting Urothelial Cancer Cells

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
L. Lilli ◽  
E. Giarnieri ◽  
S. Scardapane
Keyword(s):  
Ground Truth ◽  
Cellular Level ◽  
Diagnostic Tools ◽  
Small Scale ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Confidence Levels ◽  
Temperature Scaling ◽  
Major Shortcoming ◽  
Deep Learning Model

Deep convolutional networks have become a powerful tool for medical imaging diagnostic. In pathology, most efforts have been focused in the subfield of histology, while cytopathology (which studies diagnostic tools at the cellular level) remains underexplored. In this paper, we propose a novel deep learning model for cancer detection from urinary cytopathology screening images. We leverage recent ideas from the field of multioutput neural networks to provide a model that can efficiently train even on small-scale datasets, such as those typically found in real-world scenarios. Additionally, we argue that calibration (i.e., providing confidence levels that are aligned with the ground truth probability of an event) has been a major shortcoming of prior works, and we experiment a number of techniques to provide a well-calibrated model. We evaluate the proposed algorithm on a novel dataset, and we show that the combination of focal loss, multiple outputs, and temperature scaling provides a model that is significantly more accurate and calibrated than a baseline deep convolutional network.

scholarly journals Graph Convolutional Networks for Epigenetic State Prediction Using Both Sequence and 3D Genome Data

2019 ◽  
Author(s):  
Jack Lanchantin ◽  
Yanjun Qi
Keyword(s):  
Long Range ◽  
Dna Sequences ◽  
State Of The Art ◽  
Convolutional Network ◽  
3D Genome ◽  
Convolutional Networks ◽  
Genome Data ◽  
Epigenetic State ◽  
State Prediction ◽  
Local Sequence

AbstractPredictive models of DNA epigenetic state such as transcription factor binding are essential for understanding regulatory processes and developing gene therapies. It is known that the 3D genome, or spatial structure of DNA, is highly influential in the epigenetic state. Deep neural networks have achieved state of the art performance on epigenetic state prediction by using short windows of DNA sequences independently. These methods, however, ignore the long-range dependencies when predicting the epigenetic states because modeling the 3D genome is challenging. In this work, we introduce ChromeGCN, a graph convolutional network for epigenetic state prediction by fusing both local sequence and long-range 3D genome information. By incorporating the 3D genome, we relax the i.i.d. assumption of local windows for a better representation of DNA. ChromeGCN explicitly incorporates known long-range interactions into the modeling, allowing us to identify and interpret those important long-range dependencies in influencing epigenetic states. We show experimentally that by fusing sequential and 3D genome data using ChromeGCN, we get a significant improvement over the state-of-the-art deep learning methods as indicated by three metrics. Importantly, we show that ChromeGCN is particularly useful for identifying epigenetic effects in those DNA windows that have a high degree of interactions with other DNA windows.

scholarly journals Graph convolutional networks for epigenetic state prediction using both sequence and 3D genome data

2020 ◽  
Vol 36 (Supplement_2) ◽  
pp. i659-i667
Author(s):  
Jack Lanchantin ◽  
Yanjun Qi
Keyword(s):  
Long Range ◽  
Dna Sequences ◽  
State Of The Art ◽  
Supplementary Information ◽  
Convolutional Network ◽  
3D Genome ◽  
Convolutional Networks ◽  
Genome Data ◽  
Epigenetic State ◽  
Local Sequence

Abstract Motivation Predictive models of DNA chromatin profile (i.e. epigenetic state), such as transcription factor binding, are essential for understanding regulatory processes and developing gene therapies. It is known that the 3D genome, or spatial structure of DNA, is highly influential in the chromatin profile. Deep neural networks have achieved state of the art performance on chromatin profile prediction by using short windows of DNA sequences independently. These methods, however, ignore the long-range dependencies when predicting the chromatin profiles because modeling the 3D genome is challenging. Results In this work, we introduce ChromeGCN, a graph convolutional network for chromatin profile prediction by fusing both local sequence and long-range 3D genome information. By incorporating the 3D genome, we relax the independent and identically distributed assumption of local windows for a better representation of DNA. ChromeGCN explicitly incorporates known long-range interactions into the modeling, allowing us to identify and interpret those important long-range dependencies in influencing chromatin profiles. We show experimentally that by fusing sequential and 3D genome data using ChromeGCN, we get a significant improvement over the state-of-the-art deep learning methods as indicated by three metrics. Importantly, we show that ChromeGCN is particularly useful for identifying epigenetic effects in those DNA windows that have a high degree of interactions with other DNA windows. Availability and implementation https://github.com/QData/ChromeGCN. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

2010 ◽  
Vol 22 (2) ◽  
pp. 511-538 ◽  
Author(s):  
Srinivas C. Turaga ◽  
Joseph F. Murray ◽  
Viren Jain ◽  
Fabian Roth ◽  
Moritz Helmstaedter ◽  
...  
Keyword(s):  
Image Segmentation ◽  
Ground Truth ◽  
Image Features ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Machine Learning Approach ◽  
Segmentation Accuracy ◽  
Segmentation Algorithms ◽  
Neuronal Processes ◽  
Segmentation Problem

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

scholarly journals DNA thermodynamics shape chromosome organization and topology

2013 ◽  
Vol 41 (2) ◽  
pp. 548-553 ◽  
Author(s):  
Andrew A. Travers ◽  
Georgi Muskhelishvili
Keyword(s):  
Dna Sequence ◽  
Dna Sequences ◽  
Chromosome Organization ◽  
Topological Properties ◽  
Genetic Organization ◽  
And Topology ◽  
Dna Translocases

How much information is encoded in the DNA sequence of an organism? We argue that the informational, mechanical and topological properties of DNA are interdependent and act together to specify the primary characteristics of genetic organization and chromatin structures. Superhelicity generated in vivo, in part by the action of DNA translocases, can be transmitted to topologically sensitive regions encoded by less stable DNA sequences.

scholarly journals Efficient End-to-End Sentence-Level Lipreading with Temporal Convolutional Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 6975
Author(s):  
Tao Zhang ◽  
Lun He ◽  
Xudong Li ◽  
Guoqing Feng
Keyword(s):  
Performance Improvement ◽  
State Of The Art ◽  
Error Rates ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Sentence Level ◽  
End To End ◽  
High Level ◽  
Improved Accuracy ◽  
Talking Face

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved a high level of accuracy on large datasets and made breakthrough progress. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data and have the defects of disappearing training gradient and slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level lipreading model, using an encoder based on a 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), and a CTC objective function as the decoder. More importantly, the proposed architecture incorporates TCN as a feature learner to decode feature. It can partly eliminate the defects of RNN (LSTM, GRU) gradient disappearance and insufficient performance, and this yields notable performance improvement as well as faster convergence. Experiments show that the training and convergence speed are 50% faster than the state-of-the-art method, and improved accuracy by 2.4% on the GRID dataset.

scholarly journals DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

2021 ◽  
Author(s):  
Yanrong Ji ◽  
Zhihan Zhou ◽  
Han Liu ◽  
Ramana V Davuluri
Keyword(s):  
Dna Sequences ◽  
Regulatory Elements ◽  
Ease Of Use ◽  
Fine Tuning ◽  
Supplementary Information ◽  
Sequence Motifs ◽  
Semantic Relationship ◽  
Accurate Identification ◽  
Conserved Sequence ◽  
Genome Wide

Abstract Motivation Deciphering the language of non-coding DNA is one of the fundamental problems in genome research. Gene regulatory code is highly complex due to the existence of polysemy and distant semantic relationship, which previous informatics methods often fail to capture especially in data-scarce scenarios. Results To address this challenge, we developed a novel pre-trained bidirectional encoder representation, named DNABERT, to capture global and transferrable understanding of genomic DNA sequences based on up and downstream nucleotide contexts. We compared DNABERT to the most widely used programs for genome-wide regulatory elements prediction and demonstrate its ease of use, accuracy and efficiency. We show that the single pre-trained transformers model can simultaneously achieve state-of-the-art performance on prediction of promoters, splice sites and transcription factor binding sites, after easy fine-tuning using small task-specific labeled data. Further, DNABERT enables direct visualization of nucleotide-level importance and semantic relationship within input sequences for better interpretability and accurate identification of conserved sequence motifs and functional genetic variant candidates. Finally, we demonstrate that pre-trained DNABERT with human genome can even be readily applied to other organisms with exceptional performance. We anticipate that the pre-trained DNABERT model can be fined tuned to many other sequence analyses tasks. Availability and implementation The source code, pretrained and finetuned model for DNABERT are available at GitHub (https://github.com/jerryji1993/DNABERT). Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document