Sequence-to-function deep learning frameworks for engineered riboregulators

Abstract While synthetic biology has revolutionized our approaches to medicine, agriculture, and energy, the design of completely novel biological circuit components beyond naturally-derived templates remains challenging due to poorly understood design rules. Toehold switches, which are programmable nucleic acid sensors, face an analogous design bottleneck; our limited understanding of how sequence impacts functionality often necessitates expensive, time-consuming screens to identify effective switches. Here, we introduce Sequence-based Toehold Optimization and Redesign Model (STORM) and Nucleic-Acid Speech (NuSpeak), two orthogonal and synergistic deep learning architectures to characterize and optimize toeholds. Applying techniques from computer vision and natural language processing, we ‘un-box’ our models using convolutional filters, attention maps, and in silico mutagenesis. Through transfer-learning, we redesign sub-optimal toehold sensors, even with sparse training data, experimentally validating their improved performance. This work provides sequence-to-function deep learning frameworks for toehold selection and design, augmenting our ability to construct potent biological circuit components and precision diagnostics.

Download Full-text

Examining Deep Learning Architectures for Crime Classification and Prediction

Forecasting ◽

10.3390/forecast3040046 ◽

2021 ◽

Vol 3 (4) ◽

pp. 741-762

Author(s):

Panagiotis Stalidis ◽

Theodoros Semertzidis ◽

Petros Daras

Keyword(s):

Deep Learning ◽

State Of The Art ◽

Open Data ◽

Training Data ◽

Crime Prediction ◽

Crime Types ◽

Improved Performance ◽

Learning Architectures ◽

And Training ◽

Crime Classification

In this paper, a detailed study on crime classification and prediction using deep learning architectures is presented. We examine the effectiveness of deep learning algorithms in this domain and provide recommendations for designing and training deep learning systems for predicting crime areas, using open data from police reports. Having time-series of crime types per location as training data, a comparative study of 10 state-of-the-art methods against 3 different deep learning configurations is conducted. In our experiments with 5 publicly available datasets, we demonstrate that the deep learning-based methods consistently outperform the existing best-performing methods. Moreover, we evaluate the effectiveness of different parameters in the deep learning architectures and give insights for configuring them to achieve improved performance in crime classification and finally crime prediction.

Download Full-text

Reverse-complement parameter sharing improves deep learning models for genomics

10.1101/103663 ◽

2017 ◽

Cited By ~ 19

Author(s):

Avanti Shrikumar ◽

Peyton Greenside ◽

Anshul Kundaje

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Dna Sequence ◽

Language Processing ◽

Learning Approaches ◽

Learning Models ◽

Network Layers ◽

Reverse Complement ◽

Improved Performance

Deep learning approaches that have produced breakthrough predictive models in computer vision, speech recognition and machine translation are now being successfully applied to problems in regulatory genomics. However, deep learning architectures used thus far in genomics are often directly ported from computer vision and natural language processing applications with few, if any, domain-specific modifications. In double-stranded DNA, the same pattern may appear identically on one strand and its reverse complement due to complementary base pairing. Here, we show that conventional deep learning models that do not explicitly model this property can produce substantially different predictions on forward and reverse-complement versions of the same DNA sequence. We present four new convolutional neural network layers that leverage the reverse-complement property of genomic DNA sequence by sharing parameters between forward and reverse-complement representations in the model. These layers guarantee that forward and reverse-complement sequences produce identical predictions within numerical precision. Using experiments on simulated and in vivo transcription factor binding data, we show that our proposed architectures lead to improved performance, faster learning and cleaner internal representations compared to conventional architectures trained on the same data.

Download Full-text

Sequence-to-function deep learning frameworks for synthetic biology

10.1101/870055 ◽

2019 ◽

Cited By ~ 2

Author(s):

Jacqueline Valeri ◽

Katherine M. Collins ◽

Bianca A. Lepe ◽

Timothy K. Lu ◽

Diogo M. Camacho

Keyword(s):

Deep Learning ◽

Synthetic Biology ◽

Hot Spots ◽

Function Model ◽

Design Rules ◽

High Performing ◽

Saliency Maps ◽

Synthetic Circuit ◽

Circuit Components ◽

Learning Frameworks

AbstractWhile synthetic biology has revolutionized our approaches to medicine, agriculture, and energy, the design of novel circuit components beyond nature-inspired templates can prove itself challenging without well-established design rules. Toehold switches — programmable nucleic acid sensors — face an analogous prediction and design bottleneck: our limited understanding of how sequence impacts functionality can require expensive, time-consuming screens for effective switches. Here, we introduce the Sequence-based Toehold Optimization and Redesign Model (STORM), a deep learning architecture that applies gradient ascent to re-engineer poorly-performing toeholds. Based on a dataset of 91,534 toehold switches, we examined convolutional filters and saliency maps of sequences to interpret our sequence-to-function model, identifying hot spots where mutations change toehold effectiveness and features unique to high-performing switches. Our modeling platform provides frameworks for future toehold selection, augmenting our ability to construct potent synthetic circuit components and precision diagnostics, and enabling straightforward translation of thisin silicoworkflow to other circuitries.

Download Full-text

DLBench: a comprehensive experimental evaluation of deep learning frameworks

Cluster Computing ◽

10.1007/s10586-021-03240-4 ◽

2021 ◽

Author(s):

Radwa Elshawi ◽

Abdul Wahab ◽

Ahmed Barnawi ◽

Sherif Sakr

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Language Processing ◽

Experimental Evaluation ◽

Short Term Memory ◽

Performance Characteristics ◽

Training Time ◽

Learning Frameworks ◽

Learning Architectures

AbstractDeep Learning (DL) has achieved remarkable progress over the last decade on various tasks such as image recognition, speech recognition, and natural language processing. In general, three main crucial aspects fueled this progress: the increasing availability of large amount of digitized data, the increasing availability of affordable parallel and powerful computing resources (e.g., GPU) and the growing number of open source deep learning frameworks that facilitate and ease the development process of deep learning architectures. In practice, the increasing popularity of deep learning frameworks calls for benchmarking studies that can effectively evaluate and understand the performance characteristics of these systems. In this paper, we conduct an extensive experimental evaluation and analysis of six popular deep learning frameworks, namely, TensorFlow, MXNet, PyTorch, Theano, Chainer, and Keras, using three types of DL architectures Convolutional Neural Networks (CNN), Faster Region-based Convolutional Neural Networks (Faster R-CNN), and Long Short Term Memory (LSTM). Our experimental evaluation considers different aspects for its comparison including accuracy, training time, convergence and resource consumption patterns. Our experiments have been conducted on both CPU and GPU environments using different datasets. We report and analyze the performance characteristics of the studied frameworks. In addition, we report a set of insights and important lessons that we have learned from conducting our experiments.

Download Full-text

Automated Source Code Generation and Auto-Completion Using Deep Learning: Comparing and Discussing Current Language Model-Related Approaches

AI ◽

10.3390/ai2010001 ◽

2021 ◽

Vol 2 (1) ◽

pp. 1-16

Author(s):

Juan Cruz-Benito ◽

Sanjay Vishwakarma ◽

Francisco Martin-Fernandez ◽

Ismael Faro

Keyword(s):

Deep Learning ◽

Learning Community ◽

Programming Languages ◽

Language Processing ◽

Code Generation ◽

Language Model ◽

Language Models ◽

Stochastic Gradient Descent ◽

Network Architectures ◽

Learning Architectures

In recent years, the use of deep learning in language models has gained much attention. Some research projects claim that they can generate text that can be interpreted as human writing, enabling new possibilities in many application areas. Among the different areas related to language processing, one of the most notable in applying this type of modeling is programming languages. For years, the machine learning community has been researching this software engineering area, pursuing goals like applying different approaches to auto-complete, generate, fix, or evaluate code programmed by humans. Considering the increasing popularity of the deep learning-enabled language models approach, we found a lack of empirical papers that compare different deep learning architectures to create and use language models based on programming code. This paper compares different neural network architectures like Average Stochastic Gradient Descent (ASGD) Weight-Dropped LSTMs (AWD-LSTMs), AWD-Quasi-Recurrent Neural Networks (QRNNs), and Transformer while using transfer learning and different forms of tokenization to see how they behave in building language models using a Python dataset for code generation and filling mask tasks. Considering the results, we discuss each approach’s different strengths and weaknesses and what gaps we found to evaluate the language models or to apply them in a real programming context.

Download Full-text

Medical Text Classification Using Hybrid Deep Learning Models with Multihead Attention

Computational Intelligence and Neuroscience ◽

10.1155/2021/9425655 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Sunil Kumar Prabhakar ◽

Dong-Ok Won

Keyword(s):

Deep Learning ◽

Language Processing ◽

Text Classification ◽

Patient Information ◽

Classification Accuracy ◽

Learning Model ◽

Training Data ◽

Machine Learning Techniques ◽

Medical Text ◽

Deep Learning Model

To unlock information present in clinical description, automatic medical text classification is highly useful in the arena of natural language processing (NLP). For medical text classification tasks, machine learning techniques seem to be quite effective; however, it requires extensive effort from human side, so that the labeled training data can be created. For clinical and translational research, a huge quantity of detailed patient information, such as disease status, lab tests, medication history, side effects, and treatment outcomes, has been collected in an electronic format, and it serves as a valuable data source for further analysis. Therefore, a huge quantity of detailed patient information is present in the medical text, and it is quite a huge challenge to process it efficiently. In this work, a medical text classification paradigm, using two novel deep learning architectures, is proposed to mitigate the human efforts. The first approach is that a quad channel hybrid long short-term memory (QC-LSTM) deep learning model is implemented utilizing four channels, and the second approach is that a hybrid bidirectional gated recurrent unit (BiGRU) deep learning model with multihead attention is developed and implemented successfully. The proposed methodology is validated on two medical text datasets, and a comprehensive analysis is conducted. The best results in terms of classification accuracy of 96.72% is obtained with the proposed QC-LSTM deep learning model, and a classification accuracy of 95.76% is obtained with the proposed hybrid BiGRU deep learning model.

Download Full-text

SENTIMENT ANALYSIS FOR SARCASTIC MESSAGES IN SOCIAL MEDIA USING DEEP LEARNING TECHNIQUES - AN EMPIRICAL STUDY

INFORMATION TECHNOLOGY IN INDUSTRY ◽

10.17762/itii.v9i2.451 ◽

2021 ◽

Vol 9 (2) ◽

pp. 1051-1052

Author(s):

K. Kavitha, Et. al.

Keyword(s):

Social Media ◽

Deep Learning ◽

Empirical Study ◽

Sentiment Analysis ◽

Learning Strategies ◽

Language Processing ◽

The People ◽

Learning Techniques ◽

Key Terms ◽

Learning Frameworks

Sentiments is the term of opinion or views about any topic expressed by the people through a source of communication. Nowadays social media is an effective platform for people to communicate and it generates huge amount of unstructured details every day. It is essential for any business organization in the current era to process and analyse the sentiments by using machine learning and Natural Language Processing (NLP) strategies. Even though in recent times the deep learning strategies are becoming more familiar due to higher capabilities of performance. This paper represents an empirical study of an application of deep learning techniques in Sentiment Analysis (SA) for sarcastic messages and their increasing scope in real time. Taxonomy of the sentiment analysis in recent times and their key terms are also been highlighted in the manuscript. The survey concludes the recent datasets considered, their key contributions and the performance of deep learning model applied with its primary purpose like sarcasm detection in order to describe the efficiency of deep learning frameworks in the domain of sentimental analysis.

Download Full-text

A Review of Deep Learning Methods for Antibodies

Antibodies ◽

10.3390/antib9020012 ◽

2020 ◽

Vol 9 (2) ◽

pp. 12 ◽

Cited By ~ 4

Author(s):

Jordan Graves ◽

Jacob Byerly ◽

Eduardo Priego ◽

Naren Makkapati ◽

S. Vince Parish ◽

...

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Drug Discovery ◽

Small Molecules ◽

Language Processing ◽

Protein Design ◽

Therapeutic Drug ◽

Major Goal ◽

Antibody Design ◽

Antibody Drug

Driven by its successes across domains such as computer vision and natural language processing, deep learning has recently entered the field of biology by aiding in cellular image classification, finding genomic connections, and advancing drug discovery. In drug discovery and protein engineering, a major goal is to design a molecule that will perform a useful function as a therapeutic drug. Typically, the focus has been on small molecules, but new approaches have been developed to apply these same principles of deep learning to biologics, such as antibodies. Here we give a brief background of deep learning as it applies to antibody drug development, and an in-depth explanation of several deep learning algorithms that have been proposed to solve aspects of both protein design in general, and antibody design in particular.

Download Full-text

Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network

Applied Sciences ◽

10.3390/app10217817 ◽

2020 ◽

Vol 10 (21) ◽

pp. 7817

Author(s):

Ivana Marin ◽

Ana Kuzmanic Skelin ◽

Tamara Grujic

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Empirical Evaluation ◽

Training Data ◽

Generalization Performance ◽

Deep Model ◽

Unseen Data ◽

Regularization Techniques ◽

Learning Architectures

The main goal of any classification or regression task is to obtain a model that will generalize well on new, previously unseen data. Due to the recent rise of deep learning and many state-of-the-art results obtained with deep models, deep learning architectures have become one of the most used model architectures nowadays. To generalize well, a deep model needs to learn the training data well without overfitting. The latter implies a correlation of deep model optimization and regularization with generalization performance. In this work, we explore the effect of the used optimization algorithm and regularization techniques on the final generalization performance of the model with convolutional neural network (CNN) architecture widely used in the field of computer vision. We give a detailed overview of optimization and regularization techniques with a comparative analysis of their performance with three CNNs on the CIFAR-10 and Fashion-MNIST image datasets.

Download Full-text

Human-Like Sketch Object Recognition via Analogical Learning

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33011336 ◽

2019 ◽

Vol 33 ◽

pp. 1336-1343

Author(s):

Kezhen Chen ◽

Irina Rabkina ◽

Matthew D. McLure ◽

Kenneth D. Forbus

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Object Recognition ◽

Image Recognition ◽

Visual Representations ◽

Learning Systems ◽

Training Data ◽

Adversarial Examples ◽

Analogical Learning

Deep learning systems can perform well on some image recognition tasks. However, they have serious limitations, including requiring far more training data than humans do and being fooled by adversarial examples. By contrast, analogical learning over relational representations tends to be far more data-efficient, requiring only human-like amounts of training data. This paper introduces an approach that combines automatically constructed qualitative visual representations with analogical learning to tackle a hard computer vision problem, object recognition from sketches. Results from the MNIST dataset and a novel dataset, the Coloring Book Objects dataset, are provided. Comparison to existing approaches indicates that analogical generalization can be used to identify sketched objects from these datasets with several orders of magnitude fewer examples than deep learning systems require.

Download Full-text