scholarly journals Predicting Protein Binding Affinity With Word Embeddings and Recurrent Neural Networks

2017 ◽  
Author(s):  
Carlo Mazzaferro
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Antigenic Determinants ◽  
Underlying Structure ◽  
Specific Response ◽  
Network Architectures ◽  
Learning Approaches ◽  
Word Embeddings ◽  
Viral Pathogens ◽  
Histocompatibility Complex

AbstractAt the core of our immunological system lies a group of proteins named Major Histocompatibility Complex (MHC), to which epitopes (also proteins sometimes named antigenic determinants), bind to eliciting a response. These responses are extremely varied and of widely different nature. For instance, Killer and Helper T cells are responsible for, respectively, counteracting viral pathogens and tumorous cells. Many other types exist, but their underlying structure can be very similar due to the fact that they all are proteins and bind to the MHC receptor in a similar fashion. With this framework in mind, being able to predict with precision the structure of a protein that will elicit a specific response in the human body represents a novel computational approach to drug discovery. Although many machine learning approaches have been used, no attempt to solve this problem using Recurrent Neural Networks (RNNs) exist. We extend the current efforts in the field by applying a variety of network architectures based on RNNs and word embeddings (WE). The code is freely available and under current development at https://github.com/carlomazzaferro/mhcPreds

Recurrent Neural Networks for Edge Intelligence

2021 ◽  
Vol 54 (4) ◽  
pp. 1-38
Author(s):  
Varsha S. Lalapura ◽  
J. Amudha ◽  
Hariramn Selvamuruga Satheesh
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Model Performance ◽  
Network Architectures ◽  
Training Procedure ◽  
Resource Constrained ◽  
Compression Technique ◽  
Creative Art ◽  
Comprehensive Survey ◽  
Intelligent Models

Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence applications such as speech recognition, predictive healthcare, creative art, and so on. Although they provide accurate superior solutions, they pose a massive challenge “training havoc.” Current expansion of IoT demands intelligent models to be deployed at the edge. This is precisely to handle increasing model sizes and complex network architectures. Design efforts to meet these for greater performance have had inverse effects on portability on edge devices with real-time constraints of memory, latency, and energy. This article provides a detailed insight into various compression techniques widely disseminated in the deep learning regime. They have become key in mapping powerful RNNs onto resource-constrained devices. While compression of RNNs is the main focus of the survey, it also highlights challenges encountered while training. The training procedure directly influences model performance and compression alongside. Recent advancements to overcome the training challenges with their strengths and drawbacks are discussed. In short, the survey covers the three-step process, namely, architecture selection, efficient training process, and suitable compression technique applicable to a resource-constrained environment. It is thus one of the comprehensive survey guides a developer can adapt for a time-series problem context and an RNN solution for the edge.

scholarly journals Structured semantic knowledge can emerge automatically from predicting word sequences in child-directed speech

2017 ◽  
Author(s):  
Philip Huebner ◽  
Jon Willits
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Semantic Knowledge ◽  
Learning System ◽  
Semantic Features ◽  
Learning Approaches ◽  
Learning Mechanisms ◽  
Distributional Learning ◽  
Structured Knowledge

Previous research has suggested that distributional learning mechanisms may contribute to the acquisition of semantic knowledge. However, distributional learning mechanisms, statistical learning, and contemporary “deep learning” approaches have been criticized for being incapable of learning the kind of abstract and structured knowledge that many think is required for acquisition of semantic knowledge. In this paper, we show that recurrent neural networks, trained on noisy naturalistic speech to children, do in fact learn what appears to be abstract and structured knowledge. We trained two types of recurrent neural networks (Simple Recurrent Network, and Long Short-Term Memory) to predict word sequences in a 5-million-word corpus of speech directed to children ages 0 to 3 years old, and assessed what semantic knowledge they acquired. We found that learned internal representations are encoding various abstract grammatical and semantic features that are useful for predicting word sequences. Assessing the organization of semantic knowledge in terms of the similarity structure, we found evidence of emergent categorical and hierarchical structure in both models. We found that the LSTM and SRN are both learning very similar kinds of representations, but the LSTM achieved higher levels of performance on a quantitative evaluation. We also trained a non-recurrent neural network, Skip-gram, on the same input to compare our results to the state-of-the-art in machine learning. We found that Skip-gram achieves relatively similar performance to the LSTM, but is representing words more in terms of thematic compared to taxonomic relations, and we provide reasons why this might be the case. Our findings show that a learning system that derives abstract, distributed representations for the purpose of predicting sequential dependencies in naturalistic language may provide insight into emergence of many properties of the developing semantic system.

scholarly journals CommNets: Communicating Neural Network Architectures for Resource Constrained Systems

2019 ◽  
Vol 33 ◽  
pp. 9909-9910
Author(s):  
Prince M Abudu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Network Architecture ◽  
Resource Constraints ◽  
Constrained Systems ◽  
Detection Accuracy ◽  
Network Architectures ◽  
Operating Environments ◽  
Hidden States

Applications that require heterogeneous sensor deployments continue to face practical challenges owing to resource constraints within their operating environments (i.e. energy efficiency, computational power and reliability). This has motivated the need for effective ways of selecting a sensing strategy that maximizes detection accuracy for events of interest using available resources and data-driven approaches. Inspired by those limitations, we ask a fundamental question: whether state-of-the-art Recurrent Neural Networks can observe different series of data and communicate their hidden states to collectively solve an objective in a distributed fashion. We realize our answer by conducting a series of systematic analyses of a Communicating Recurrent Neural Network architecture on varying time-steps, objective functions and number of nodes. The experimental setup we employ models tasks synonymous with those in Wireless Sensor Networks. Our contributions show that Recurrent Neural Networks can communicate through their hidden states and we achieve promising results.

Generation of Realistic Navigation Paths for Web Site Testing Using RNN and GAN

2021 ◽  
Author(s):  
Silvio Pavanetto ◽  
Marco Brambilla
Keyword(s):  
Neural Networks ◽  
Web Sites ◽  
Recurrent Neural Networks ◽  
Lower Cost ◽  
Automated Testing ◽  
Learning Approaches ◽  
Site Testing ◽  
Model Driven ◽  
Program Testing ◽  
Development Tools

For applications that have not yet been launched, a reliable way for creating online navigation logs may be crucial, enabling developers to test their products as though they were being used by real users. This might lead to faster and lower-cost program testing and enhancement, especially in terms of usability and interaction. In this work we propose a method for using deep learning approaches such as recurrent neural networks (RNN) and generative adversarial neural networks (GANN) to produce high-quality weblogs. Eventually, we can utilize the created data for automated testing and improvement of Web sites prior to their release with the aid of model-driven development tools such as IFML Editor.

scholarly journals Creating a robot localization monitor using particle filter and machine learning approaches

2021 ◽  
Author(s):  
Matthias Eder ◽  
Michael Reip ◽  
Gerald Steinbauer
Keyword(s):  
Neural Networks ◽  
Particle Filter ◽  
Recurrent Neural Networks ◽  
Position Estimation ◽  
Robot Localization ◽  
Learning Approaches ◽  
Localization Accuracy ◽  
Robot Systems ◽  
Localization Performance ◽  
Localization Approach

AbstractRobot localization is a fundamental capability of all mobile robots. Because of uncertainties in acting and sensing, and environmental factors such as people flocking around robots, there is always the risk that a robot loses its localization. Very often behaviors of robots rely on a reliable position estimation. Thus, for dependability of robot systems it is of great interest for the system to know the state of its localization component. In this paper we present an approach that allows a robot to asses if the localization is still correct. The approach assumes that the underlying localization approach is based on a particle filter. We use deep learning to identify temporal patterns in the particles in the case of losing/lost localization. These patterns are then combined with weak classifiers from the particle set and sensor perception for boosted learning of a localization estimator. Through the extraction of features generated by neural networks and its usage for training strong classifiers, the robots localization accuracy can be estimated. The approach is evaluated in a simulated transport robot environment where a degraded localization is provoked by disturbances cased by dynamic obstacles. Results show that it is possible to monitor the robots localization accuracy using convolutional as well as recurrent neural networks. The additional boosting using Adaboost also yields an increase in training accuracy. Thus, this paper directly contributes to the verification of localization performance.

scholarly journals Leveraging Deep Learning for Visual Odometry Using Optical Flow

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1313
Author(s):  
Tejas Pandey ◽  
Dexmont Pena ◽  
Jonathan Byrne ◽  
David Moloney
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Optical Flow ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Visual Odometry ◽  
Camera Motion ◽  
Learning Approaches ◽  
Outlier Rejection ◽  
Entire Trajectory

In this paper, we study deep learning approaches for monocular visual odometry (VO). Deep learning solutions have shown to be effective in VO applications, replacing the need for highly engineered steps, such as feature extraction and outlier rejection in a traditional pipeline. We propose a new architecture combining ego-motion estimation and sequence-based learning using deep neural networks. We estimate camera motion from optical flow using Convolutional Neural Networks (CNNs) and model the motion dynamics using Recurrent Neural Networks (RNNs). The network outputs the relative 6-DOF camera poses for a sequence, and implicitly learns the absolute scale without the need for camera intrinsics. The entire trajectory is then integrated without any post-calibration. We evaluate the proposed method on the KITTI dataset and compare it with traditional and other deep learning approaches in the literature.

Physical Exercise Recommendation and Success Prediction Using Interconnected Recurrent Neural Networks (Preprint)

2021 ◽  
Author(s):  
Arash Mahyari ◽  
Peter Pirolli
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Computational Models ◽  
Recommendation System ◽  
Developed Countries ◽  
Training Data ◽  
Published Data ◽  
Large Sample Size ◽  
Learning Approaches ◽  
Success Rates

BACKGROUND Unhealthy behaviors, e.g., physical inactivity and unhealthful food choice, are the primary healthcare cost drivers in developed countries. Pervasive computational, sensing, and communication technology provided by smartphones and smartwatches have made it possible to support individuals in their everyday lives to develop healthier lifestyles. OBJECTIVE This paper proposes an exercise recommendation system to recommend daily exercises to elderly population. METHODS The system, consisting of two inter-connected recurrent neural networks (RNNs), uses the history of workouts to recommend the next workout activity for each individual. The system then predicts the probability of successful completion of the predicted activity by the individual. RESULTS The prediction accuracy of this interconnected-RNN model is assessed on previously published data from a four-week mobile health experiment and is shown to improve upon previous predictions from a computational cognitive model. The proposed system is able to predict the next exercise for each individual with 80% accuracy. CONCLUSIONS The dual-RNN system for recommending workout exercises along with predicting individual success rates achieves high accuracy for individuals from whom we do not have any training data. The proposed system was validated this achievement by training the proposed model on a set of users and testing on a new set of test users. Future studies will involve combinations of explanatory computational models such as ACT-R and machine learning approaches such as the dual-RNN system to address the shortcomings of existing recommendations systems in need of large sample size.

Sign in / Sign up

Export Citation Format

Share Document