Explainable Transformer Models for Functional Genomics in Prokaryotes

AbstractThe effectiveness of deep learning methods can be largely attributed to the automated extraction of relevant features from raw data. In the field of functional genomics, this generally comprises the automatic selection of relevant nucleotide motifs from DNA sequences. To benefit from automated learning methods, new strategies are required that unveil the decision-making process of trained models. In this paper, we present several methods that can be used to gather insights on biological processes that drive any genome annotation task. This work builds upon a transformer-based neural network framework designed for prokaryotic genome annotation purposes. We find that the majority of sub-units (attention heads) of the model are specialized towards identifying DNA binding sites. Working with a neural network trained to detect transcription start sites in E. coli, we successfully characterize both locations and consensus sequences of transcription factor binding sites, including both well-known and potentially novel elements involved in the initiation of the transcription process.

Download Full-text

Explainability in transformer models for functional genomics

Briefings in Bioinformatics ◽

10.1093/bib/bbab060 ◽

2021 ◽

Author(s):

Jim Clauwaert ◽

Gerben Menschaert ◽

Willem Waegeman

Keyword(s):

Functional Genomics ◽

Dna Sequences ◽

Prokaryotic Genome ◽

Learning Methods ◽

New Approach ◽

Consensus Sequences ◽

Automated Learning ◽

Transcription Process ◽

New Strategies ◽

Selection Of

Abstract The effectiveness of deep learning methods can be largely attributed to the automated extraction of relevant features from raw data. In the field of functional genomics, this generally concerns the automatic selection of relevant nucleotide motifs from DNA sequences. To benefit from automated learning methods, new strategies are required that unveil the decision-making process of trained models. In this paper, we present a new approach that has been successful in gathering insights on the transcription process in Escherichia coli. This work builds upon a transformer-based neural network framework designed for prokaryotic genome annotation purposes. We find that the majority of subunits (attention heads) of the model are specialized towards identifying transcription factors and are able to successfully characterize both their binding sites and consensus sequences, uncovering both well-known and potentially novel elements involved in the initiation of the transcription process. With the specialization of the attention heads occurring automatically, we believe transformer models to be of high interest towards the creation of explainable neural networks in this field.

Download Full-text

ANFIS-based fuzzy systems for searching dna-protein binding sites

10.1101/058800 ◽

2016 ◽

Cited By ~ 2

Author(s):

Dianhui Wang ◽

Monther Alhamdoosh ◽

Witold Pedrycz

Keyword(s):

Dna Sequences ◽

Binding Sites ◽

Fuzzy Inference ◽

Machine Learning Techniques ◽

Inference System ◽

Learner Model ◽

Transcription Start Sites ◽

Reduction Techniques ◽

Dna Binding Sites ◽

Inference Techniques

AbstractTranscriptional regulation mainly controls how genes are expressed and how cells behave based on the transcription factor (TF) proteins that bind upstream of the transcription start sites (TSSs) of genes. These TF DNA binding sites (TFBSs) are usually short (5-15 base pairs) and degenerate (some positions can have multiple possible alternatives). Traditionally, computational methods scan DNA sequences using the position weight matrix (PWM) of a given TF, calculate binding scores for each K-mer against the PWM, and finally classify a K-mer as to whether it is a putative TFBS or a background sequence based on a cut-off threshold. The FSCAN system, which is proposed in this paper, employs machine learning techniques to build a learner model that is able to identify TFBSs in a set of bound sequences without the need for a cut-off threshold. Our proposed method utilizes fuzzy inference techniques along with a distribution-based filtering algorithm to predict the binding sites of a TF given its PWM model and phastCons scores for the input DNA sequences. Data imbalance reduction techniques are also used to ease the learning of the adaptive-neuro fuzzy inference system (ANFIS) algorithm. The proposed system is tested on 22 ChIP-chip sequence-sets from the Saccharomyces Cerevisiae genome. Our results show that FSCAN outperforms other approaches like MatInspector and MATCH and is quite robust. As more transcriptional data becomes available, our proposed framework encourages the use of fuzzy logic techniques in the prediction of TFBSs.

Download Full-text

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189290 ◽

2020 ◽

Vol 39 (6) ◽

pp. 8927-8935

Author(s):

Bing Zheng ◽

Dawei Yun ◽

Yan Liang

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Behavior Pattern ◽

Rapid Development ◽

Video Data ◽

Support Vector ◽

Behavior Recognition ◽

Learning Methods ◽

The Impact ◽

Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

Download Full-text

Faculty Opinions recommendation of Prokka: rapid prokaryotic genome annotation.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.718312026.793492408 ◽

2014 ◽

Author(s):

Stephen Turner

Keyword(s):

Genome Annotation ◽

Prokaryotic Genome

Download Full-text

Sequence-Based Deep Learning Frameworks on Enhancer-Promoter Interactions Prediction

Current Pharmaceutical Design ◽

10.2174/1381612826666201124112710 ◽

2020 ◽

Vol 26 ◽

Author(s):

Xiaoping Min ◽

Fengqing Lu ◽

Chunyan Li

Keyword(s):

Gene Expression ◽

Deep Learning ◽

Dna Sequences ◽

Experimental Methods ◽

Learning Methods ◽

Comprehensive Review ◽

Genomic Features ◽

Disease Mechanisms ◽

Evaluation Strategies ◽

Learning Frameworks

: Enhancer-promoter interactions (EPIs) in the human genome are of great significance to transcriptional regulation which tightly controls gene expression. Identification of EPIs can help us better deciphering gene regulation and understanding disease mechanisms. However, experimental methods to identify EPIs are constrained by the fund, time and manpower while computational methods using DNA sequences and genomic features are viable alternatives. Deep learning methods have shown promising prospects in classification and efforts that have been utilized to identify EPIs. In this survey, we specifically focus on sequence-based deep learning methods and conduct a comprehensive review of the literatures of them. We first briefly introduce existing sequence-based frameworks on EPIs prediction and their technique details. After that, we elaborate on the dataset, pre-processing means and evaluation strategies. Finally, we discuss the challenges these methods are confronted with and suggest several future opportunities.

Download Full-text

A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Electronics ◽

10.3390/electronics10010081 ◽

2021 ◽

Vol 10 (1) ◽

pp. 81

Author(s):

Jianbin Xiong ◽

Dezheng Yu ◽

Shuangyin Liu ◽

Lei Shu ◽

Xiaochan Wang ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Plant Species ◽

Image Recognition ◽

Recurrent Neural Network ◽

Plant Diseases ◽

Learning Methods ◽

Smart Agriculture ◽

Important Branch

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

Download Full-text

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

Entropy ◽

10.3390/e23060667 ◽

2021 ◽

Vol 23 (6) ◽

pp. 667

Author(s):

Wei Chen ◽

Qiang Sun ◽

Xiaomin Chen ◽

Gangcai Xie ◽

Huiqun Wu ◽

...

Keyword(s):

Neural Network ◽

Systematic Review ◽

Deep Learning ◽

Heart Sounds ◽

Learning Approaches ◽

Automated Classification ◽

Learning Methods ◽

Artificial Intelligence Technology ◽

Medical Big Data ◽

Effective Models

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Download Full-text

Possibility of Autonomous Estimation of Shiba Goat’s Estrus and Non-Estrus Behavior by Machine Learning Methods

Animals ◽

10.3390/ani10050771 ◽

2020 ◽

Vol 10 (5) ◽

pp. 771

Author(s):

Toshiya Arakawa

Keyword(s):

Neural Network ◽

Machine Learning ◽

Random Forest ◽

Markov Models ◽

Tracking System ◽

Video Tracking ◽

Training Data ◽

Support Vector ◽

Learning Methods ◽

Machine Learning Methods

Mammalian behavior is typically monitored by observation. However, direct observation requires a substantial amount of effort and time, if the number of mammals to be observed is sufficiently large or if the observation is conducted for a prolonged period. In this study, machine learning methods as hidden Markov models (HMMs), random forests, support vector machines (SVMs), and neural networks, were applied to detect and estimate whether a goat is in estrus based on the goat’s behavior; thus, the adequacy of the method was verified. Goat’s tracking data was obtained using a video tracking system and used to estimate whether they, which are in “estrus” or “non-estrus”, were in either states: “approaching the male”, or “standing near the male”. Totally, the PC of random forest seems to be the highest. However, The percentage concordance (PC) value besides the goats whose data were used for training data sets is relatively low. It is suggested that random forest tend to over-fit to training data. Besides random forest, the PC of HMMs and SVMs is high. However, considering the calculation time and HMM’s advantage in that it is a time series model, HMM is better method. The PC of neural network is totally low, however, if the more goat’s data were acquired, neural network would be an adequate method for estimation.

Download Full-text

Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals

Sensors ◽

10.3390/s19010210 ◽

2019 ◽

Vol 19 (1) ◽

pp. 210 ◽

Cited By ~ 32

Author(s):

Zied Tayeb ◽

Juri Fedjaev ◽

Nejla Ghaboosi ◽

Christoph Richter ◽

Lukas Everding ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Convolutional Neural Network ◽

Motor Imagery ◽

Classification Performance ◽

Feature Engineering ◽

Learning Models ◽

Eeg Signals ◽

Learning Methods

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from “BCI Competition IV”. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

Download Full-text