Physical Exercise Recommendation and Success Prediction Using Interconnected Recurrent Neural Networks (Preprint)

2021 ◽  
Author(s):  
Arash Mahyari ◽  
Peter Pirolli
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Computational Models ◽  
Recommendation System ◽  
Developed Countries ◽  
Training Data ◽  
Published Data ◽  
Large Sample Size ◽  
Learning Approaches ◽  
Success Rates

BACKGROUND Unhealthy behaviors, e.g., physical inactivity and unhealthful food choice, are the primary healthcare cost drivers in developed countries. Pervasive computational, sensing, and communication technology provided by smartphones and smartwatches have made it possible to support individuals in their everyday lives to develop healthier lifestyles. OBJECTIVE This paper proposes an exercise recommendation system to recommend daily exercises to elderly population. METHODS The system, consisting of two inter-connected recurrent neural networks (RNNs), uses the history of workouts to recommend the next workout activity for each individual. The system then predicts the probability of successful completion of the predicted activity by the individual. RESULTS The prediction accuracy of this interconnected-RNN model is assessed on previously published data from a four-week mobile health experiment and is shown to improve upon previous predictions from a computational cognitive model. The proposed system is able to predict the next exercise for each individual with 80% accuracy. CONCLUSIONS The dual-RNN system for recommending workout exercises along with predicting individual success rates achieves high accuracy for individuals from whom we do not have any training data. The proposed system was validated this achievement by training the proposed model on a set of users and testing on a new set of test users. Future studies will involve combinations of explanatory computational models such as ACT-R and machine learning approaches such as the dual-RNN system to address the shortcomings of existing recommendations systems in need of large sample size.

scholarly journals Transfer Learning with Deep Recurrent Neural Networks for Remaining Useful Life Estimation

2018 ◽  
Vol 8 (12) ◽  
pp. 2416 ◽  
Author(s):  
Ansi Zhang ◽  
Honglei Wang ◽  
Shaobo Li ◽  
Yuxin Cui ◽  
Zhonghao Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Transfer Learning ◽  
Recurrent Neural Networks ◽  
Prediction Models ◽  
Learning Algorithm ◽  
Remaining Useful Life ◽  
Operating Conditions ◽  
Training Data ◽  
Good Prediction ◽  
Useful Life

Prognostics, such as remaining useful life (RUL) prediction, is a crucial task in condition-based maintenance. A major challenge in data-driven prognostics is the difficulty of obtaining a sufficient number of samples of failure progression. However, for traditional machine learning methods and deep neural networks, enough training data is a prerequisite to train good prediction models. In this work, we proposed a transfer learning algorithm based on Bi-directional Long Short-Term Memory (BLSTM) recurrent neural networks for RUL estimation, in which the models can be first trained on different but related datasets and then fine-tuned by the target dataset. Extensive experimental results show that transfer learning can in general improve the prediction models on the dataset with a small number of samples. There is one exception that when transferring from multi-type operating conditions to single operating conditions, transfer learning led to a worse result.

Locally Recurrent Neural Networks and Their Applications

2010 ◽  
pp. 195-222
Author(s):  
Todor D. Ganchev
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Computational Models ◽  
Machine Learning Techniques ◽  
Spatial Correlations ◽  
Temporal Correlations ◽  
Learning Techniques ◽  
Locally Recurrent ◽  
The One ◽  
And Control

In this chapter we review various computational models of locally recurrent neurons and deliberate the architecture of some archetypal locally recurrent neural networks (LRNNs) that are based on them. Generalizations of these structures are discussed as well. Furthermore, we point at a number of realworld applications of LRNNs that have been reported in past and recent publications. These applications involve classification or prediction of temporal sequences, discovering and modeling of spatial and temporal correlations, process identification and control, etc. Validation experiments reported in these developments provide evidence that locally recurrent architectures are capable of identifying and exploiting temporal and spatial correlations (i.e., the context in which events occur), which is the main reason for their advantageous performance when compared with the one of their non-recurrent counterparts or other reasonable machine learning techniques.

scholarly journals A Novel Method for Training an Echo State Network with Feedback-Error Learning

2013 ◽  
Vol 2013 ◽  
pp. 1-9
Author(s):  
Rikke Amilde Løvlid
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Training Data ◽  
Echo State Network ◽  
Training Method ◽  
Computationally Efficient ◽  
Echo State Networks ◽  
Feedback Error Learning ◽  
Feedback Error ◽  
Error Learning

Echo state networks are a relatively new type of recurrent neural networks that have shown great potentials for solving non-linear, temporal problems. The basic idea is to transform the low dimensional temporal input into a higher dimensional state, and then train the output connection weights to make the system output the target information. Because only the output weights are altered, training is typically quick and computationally efficient compared to training of other recurrent neural networks. This paper investigates using an echo state network to learn the inverse kinematics model of a robot simulator with feedback-error-learning. In this scheme teacher forcing is not perfect, and joint constraints on the simulator makes the feedback error inaccurate. A novel training method which is less influenced by the noise in the training data is proposed and compared to the traditional ESN training method.

scholarly journals Sample-level sound synthesis with recurrent neural networks and conceptors

2018 ◽  
Author(s):  
Chris Kiefer
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Frequency Control ◽  
Synthesis Method ◽  
Sound Synthesis ◽  
Boolean Logic ◽  
Training Data ◽  
Reservoir Computing ◽  
Research Questions ◽  
Granular Synthesis

Conceptors are a recent development in the field of reservoir computing; they can be used to influence the dynamics of recurrent neural networks (RNNs), enabling generation of arbitrary patterns based on training data. Conceptors allow interpolation and extrapolation between patterns, and also provide a system of boolean logic for combining patterns together. Generation and manipulation of arbitrary patterns using conceptors has significant potential as a sound synthesis method for applications in computer music and procedural audio but has yet to be explored. Two novel methods of sound synthesis based on conceptors are introduced. Conceptular Synthesis is based on granular synthesis; sets of conceptors are trained to recall varying patterns from a single RNN, then a runtime mechanism switches between them, generating short patterns which are recombined into a longer sound. Conceptillators are trainable, pitch-controlled oscillators for harmonically rich waveforms, commonly used in a variety of sound synthesis applications. Both systems can exploit conceptor pattern morphing, boolean logic and manipulation of RNN dynamics, enabling new creative sonic possibilities. Experiments reveal how RNN runtime parameters can be used for pitch-independent timestretching and for precise frequency control of cyclic waveforms. They show how these techniques can create highly malleable sound synthesis models, trainable using short sound samples. Limitations are revealed with regards to reproduction quality, and pragmatic limitations are also shown, where exponential rises in computation and memory requirements preclude the use of these models for training with longer sound samples. The techniques presented here represent an initial exploration of the sound synthesis potential of conceptors; future possibilities and research questions are outlined, including possibilities in generative sound.

scholarly journals Accelerating recommendation system training by leveraging popular choices

2021 ◽  
Vol 15 (1) ◽  
pp. 127-140
Author(s):  
Muhammad Adnan ◽  
Yassaman Ebrahimzadeh Maboud ◽  
Divya Mahajan ◽  
Prashant J. Nair
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Recommendation System ◽  
Training Data ◽  
Categorical Variables ◽  
Numerical Representation ◽  
Data Layout ◽  
Production Scale ◽  
Training Time ◽  
Usage Patterns

Recommender models are commonly used to suggest relevant items to a user for e-commerce and online advertisement-based applications. These models use massive embedding tables to store numerical representation of items' and users' categorical variables (memory intensive) and employ neural networks (compute intensive) to generate final recommendations. Training these large-scale recommendation models is evolving to require increasing data and compute resources. The highly parallel neural networks portion of these models can benefit from GPU acceleration however, large embedding tables often cannot fit in the limited-capacity GPU device memory. Hence, this paper deep dives into the semantics of training data and obtains insights about the feature access, transfer, and usage patterns of these models. We observe that, due to the popularity of certain inputs, the accesses to the embeddings are highly skewed with a few embedding entries being accessed up to 10000X more. This paper leverages this asymmetrical access pattern to offer a framework, called FAE, and proposes a hot-embedding aware data layout for training recommender models. This layout utilizes the scarce GPU memory for storing the highly accessed embeddings, thus reduces the data transfers from CPU to GPU. At the same time, FAE engages the GPU to accelerate the executions of these hot embedding entries. Experiments on production-scale recommendation models with real datasets show that FAE reduces the overall training time by 2.3X and 1.52X in comparison to XDL CPU-only and XDL CPU-GPU execution while maintaining baseline accuracy.

scholarly journals Applying Deep Neural Networks and Ensemble Machine Learning Methods to Forecast Airborne Ambrosia Pollen

2019 ◽  
Vol 16 (11) ◽  
pp. 1992 ◽  
Author(s):  
Gebreab K. Zewdie ◽  
David J. Lary ◽  
Estelle Levetin ◽  
Gemechu F. Garuma
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Land Surface ◽  
Deep Neural Networks ◽  
Airborne Pollen ◽  
Training Data ◽  
Gradient Boosting ◽  
Learning Approaches ◽  
Ambrosia Pollen ◽  
Extreme Gradient Boosting

Allergies to airborne pollen are a significant issue affecting millions of Americans. Consequently, accurately predicting the daily concentration of airborne pollen is of significant public benefit in providing timely alerts. This study presents a method for the robust estimation of the concentration of airborne Ambrosia pollen using a suite of machine learning approaches including deep learning and ensemble learners. Each of these machine learning approaches utilize data from the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric weather and land surface reanalysis. The machine learning approaches used for developing a suite of empirical models are deep neural networks, extreme gradient boosting, random forests and Bayesian ridge regression methods for developing our predictive model. The training data included twenty-four years of daily pollen concentration measurements together with ECMWF weather and land surface reanalysis data from 1987 to 2011 is used to develop the machine learning predictive models. The last six years of the dataset from 2012 to 2017 is used to independently test the performance of the machine learning models. The correlation coefficients between the estimated and actual pollen abundance for the independent validation datasets for the deep neural networks, random forest, extreme gradient boosting and Bayesian ridge were 0.82, 0.81, 0.81 and 0.75 respectively, showing that machine learning can be used to effectively forecast the concentrations of airborne pollen.

Partial AUC Optimisation Using Recurrent Neural Networks for Music Detection with Limited Training Data

2020 ◽  
Author(s):  
Pablo Gimeno ◽  
Victoria Mingote ◽  
Alfonso Ortega ◽  
Antonio Miguel ◽  
Eduardo Lleida
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Training Data ◽  
Partial Auc

Learning Brain Dynamics with Coupled Low-Dimensional Nonlinear Oscillators and Deep Recurrent Networks

2021 ◽  
pp. 1-40
Author(s):  
Germán Abrevaya ◽  
Guillaume Dumas ◽  
Aleksandr Y. Aravkin ◽  
Peng Zheng ◽  
Jean-Christophe Gagnon-Audet ◽  
...  
Keyword(s):  
Neural Networks ◽  
Dynamical Systems ◽  
Brain Imaging ◽  
Recurrent Neural Networks ◽  
Practical Importance ◽  
Parameters Estimation ◽  
Training Data ◽  
Autoregressive Models ◽  
Unseen Data ◽  
Low Dimensional

Abstract Many natural systems, especially biological ones, exhibit complex multivariate nonlinear dynamical behaviors that can be hard to capture by linear autoregressive models. On the other hand, generic nonlinear models such as deep recurrent neural networks often require large amounts of training data, not always available in domains such as brain imaging; also, they often lack interpretability. Domain knowledge about the types of dynamics typically observed in such systems, such as a certain type of dynamical systems models, could complement purely data-driven techniques by providing a good prior. In this work, we consider a class of ordinary differential equation (ODE) models known as van der Pol (VDP) oscil lators and evaluate their ability to capture a low-dimensional representation of neural activity measured by different brain imaging modalities, such as calcium imaging (CaI) and fMRI, in different living organisms: larval zebrafish, rat, and human. We develop a novel and efficient approach to the nontrivial problem of parameters estimation for a network of coupled dynamical systems from multivariate data and demonstrate that the resulting VDP models are both accurate and interpretable, as VDP's coupling matrix reveals anatomically meaningful excitatory and inhibitory interactions across different brain subsystems. VDP outperforms linear autoregressive models (VAR) in terms of both the data fit accuracy and the quality of insight provided by the coupling matrices and often tends to generalize better to unseen data when predicting future brain activity, being comparable to and sometimes better than the recurrent neural networks (LSTMs). Finally, we demonstrate that our (generative) VDP model can also serve as a data-augmentation tool leading to marked improvements in predictive accuracy of recurrent neural networks. Thus, our work contributes to both basic and applied dimensions of neuroimaging: gaining scientific insights and improving brain-based predictive models, an area of potentially high practical importance in clinical diagnosis and neurotechnology.

scholarly journals Deep learning in Emergency Medicine: Recent contributions and methodological challenges

2020 ◽  
Vol 16 (1) ◽  
Author(s):  
Francesco Faita
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Emergency Medicine ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Recurrent Neural Networks ◽  
Computational Models ◽  
Clinical Usefulness ◽  
Human Knowledge ◽  
Deep Convolutional Neural Networks

In the last few years, artificial intelligence (AI) technology has grown dramatically impacting several fields of human knowledge and medicine in particular. Among other approaches, deep learning, which is a subset of AI based on specific computational models, such as deep convolutional neural networks and recurrent neural networks, has shown exceptional performance in images and signals processing. Accordingly, emergency medicine will benefit from the adoption of this technology. However, a particular attention should be devoted to the review of these papers in order to exclude overoptimistic results from clinically transferable ones. We presented a group of studies recently published on PubMed and selected by keywords ‘deep learning emergency medicine’ and ‘artificial intelligence emergency medicine’ with the aim of highlighting their methodological strengths and weaknesses, as well as their clinical usefulness.

scholarly journals Structured semantic knowledge can emerge automatically from predicting word sequences in child-directed speech

2017 ◽  
Author(s):  
Philip Huebner ◽  
Jon Willits
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Semantic Knowledge ◽  
Learning System ◽  
Semantic Features ◽  
Learning Approaches ◽  
Learning Mechanisms ◽  
Distributional Learning ◽  
Structured Knowledge

Previous research has suggested that distributional learning mechanisms may contribute to the acquisition of semantic knowledge. However, distributional learning mechanisms, statistical learning, and contemporary “deep learning” approaches have been criticized for being incapable of learning the kind of abstract and structured knowledge that many think is required for acquisition of semantic knowledge. In this paper, we show that recurrent neural networks, trained on noisy naturalistic speech to children, do in fact learn what appears to be abstract and structured knowledge. We trained two types of recurrent neural networks (Simple Recurrent Network, and Long Short-Term Memory) to predict word sequences in a 5-million-word corpus of speech directed to children ages 0 to 3 years old, and assessed what semantic knowledge they acquired. We found that learned internal representations are encoding various abstract grammatical and semantic features that are useful for predicting word sequences. Assessing the organization of semantic knowledge in terms of the similarity structure, we found evidence of emergent categorical and hierarchical structure in both models. We found that the LSTM and SRN are both learning very similar kinds of representations, but the LSTM achieved higher levels of performance on a quantitative evaluation. We also trained a non-recurrent neural network, Skip-gram, on the same input to compare our results to the state-of-the-art in machine learning. We found that Skip-gram achieves relatively similar performance to the LSTM, but is representing words more in terms of thematic compared to taxonomic relations, and we provide reasons why this might be the case. Our findings show that a learning system that derives abstract, distributed representations for the purpose of predicting sequential dependencies in naturalistic language may provide insight into emergence of many properties of the developing semantic system.

Sign in / Sign up

Export Citation Format

Share Document