scholarly journals Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation

2019 ◽  
Author(s):  
Zeping Yu ◽  
Jianxun Lian ◽  
Ahmad Mahmoody ◽  
Gongshen Liu ◽  
Xing Xie
Keyword(s):  
User Modeling ◽  
Short Term Memory ◽  
Contextual Information ◽  
Personalized Recommendation ◽  
Short Term ◽  
The Past ◽  
Visual Computing ◽  
Long Short Term Memory ◽  
Natural Way

User modeling is an essential task for online recommender systems. In the past few decades, collaborative filtering (CF) techniques have been well studied to model users' long term preferences. Recently, recurrent neural networks (RNN) have shown a great advantage in modeling users' short term preference. A natural way to improve the recommender is to combine both long-term and short-term modeling. Previous approaches neglect the importance of dynamically integrating these two user modeling paradigms. Moreover, users' behaviors are much more complex than sentences in language modeling or images in visual computing, thus the classical structures of RNN such as Long Short-Term Memory (LSTM) need to be upgraded for better user modeling. In this paper, we improve the traditional RNN structure by proposing a time-aware controller and a content-aware controller, so that contextual information can be well considered to control the state transition. We further propose an attention-based framework to combine users' long-term and short-term preferences, thus users' representation can be generated adaptively according to the specific context. We conduct extensive experiments on both public and industrial datasets. The results demonstrate that our proposed method outperforms several state-of-art methods consistently.

Sequence Model based Cloudburst Prediction for the Indian State of Uttarakhand

2021 ◽  
Vol 14 (7) ◽  
pp. 1-9
Author(s):  
M. Sivagami ◽  
P. Radha ◽  
A. Balasundaram
Keyword(s):  
Predictive Power ◽  
Short Term Memory ◽  
Geographic Location ◽  
Short Term ◽  
The Past ◽  
Learning Techniques ◽  
Indian State ◽  
Long Short Term Memory ◽  
Life Challenge ◽  
Gated Recurrent Unit

Predicting the phenomenon of cloudburst has been a larger than life challenge to many weather and rain scientists. The very nature of cloudburst occurrence itself complicates the prediction of cloudburst. Since, cloudburst downpour occurs over a short span of time and is confined to very narrow geographic location, it is highly difficult for weather scientists to make any cloudburst predictions. In this work, the authors propose a cloudburst prediction model that leverages deep learning techniques to predict the occurrence of cloudburst in a location. The authors have collected the data pertaining to the cloudburst events that have occurred in the Indian State of Uttarakhand over the past decade and developed the model. Experiments were conducted using time series sequence models namely Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). Predictive Power Score (PPS) has been used to extract the essential features that are fed as input to these sequence models. The performance of sequence models has been discussed in terms of loss function and accuracy and the results are promising for GRU based model in comparison with other sequence models.

scholarly journals Long Short Term Memory Recurrent Network for Standard and Poor’s 500 Index Modelling

2018 ◽  
Vol 7 (4.15) ◽  
pp. 25 ◽  
Author(s):  
Said Jadid Abdulkadir ◽  
Hitham Alhussian ◽  
Muhammad Nazmi ◽  
Asim A Elsheikh
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Forecasting time-series data are imperative especially when planning is required through modelling using uncertain knowledge of future events. Recurrent neural network models have been applied in the industry and outperform standard artificial neural networks in forecasting, but fail in long term time-series forecasting due to the vanishing gradient problem. This study offers a robust solution that can be implemented for long-term forecasting using a special architecture of recurrent neural network known as Long Short Term Memory (LSTM) model to overcome the vanishing gradient problem. LSTM is specially designed to avoid the long-term dependency problem as their default behavior. Empirical analysis is performed using quantitative forecasting metrics and comparative model performance on the forecasted outputs. An evaluation analysis is performed to validate that the LSTM model provides better forecasted outputs on Standard & Poor’s 500 Index (S&P 500) in terms of error metrics as compared to other forecasting models.  

scholarly journals Deep Neural Network and Long Short-Term Memory for Electric Power Load Forecasting

2020 ◽  
Vol 10 (18) ◽  
pp. 6489
Author(s):  
Namrye Son ◽  
Seunghak Yang ◽  
Jeongseung Na
Keyword(s):  
Power Consumption ◽  
Short Term Memory ◽  
Load Forecasting ◽  
Power Demand ◽  
Short Term ◽  
Power Load ◽  
Short And Long Term ◽  
Long Short Term Memory ◽  
Electric Power Load

Forecasting domestic and foreign power demand is crucial for planning the operation and expansion of facilities. Power demand patterns are very complex owing to energy market deregulation. Therefore, developing an appropriate power forecasting model for an electrical grid is challenging. In particular, when consumers use power irregularly, the utility cannot accurately predict short- and long-term power consumption. Utilities that experience short- and long-term power demands cannot operate power supplies reliably; in worst-case scenarios, blackouts occur. Therefore, the utility must predict the power demands by analyzing the customers’ power consumption patterns for power supply stabilization. For this, a medium- and long-term power forecasting is proposed. The electricity demand forecast was divided into medium-term and long-term load forecast for customers with different power consumption patterns. Among various deep learning methods, deep neural networks (DNNs) and long short-term memory (LSTM) were employed for the time series prediction. The DNN and LSTM performances were compared to verify the proposed model. The two models were tested, and the results were examined with the accuracies of the six most commonly used evaluation measures in the medium- and long-term electric power load forecasting. The DNN outperformed the LSTM, regardless of the customer’s power pattern.

scholarly journals Learning Robot Speech Models to Predict Speech Acts in HRI

2016 ◽  
Vol 9 (1) ◽  
pp. 295-306
Author(s):  
Ankuj Arora ◽  
Humbert Fiorino ◽  
Damien Pellier ◽  
Sylvie Pesty
Keyword(s):  
Speech Acts ◽  
Short Term Memory ◽  
The State ◽  
Short Term ◽  
Long Run ◽  
Action Sequences ◽  
Long Short Term Memory ◽  
Max Sat ◽  
Speech Model

Abstract In order to be acceptable and able to “camouflage” into their physio-social context in the long run, robots need to be not just functional, but autonomously psycho-affective as well. This motivates a long term necessity of introducing behavioral autonomy in robots, so they can autonomously communicate with humans without the need of “wizard” intervention. This paper proposes a technique to learn robot speech models from human-robot dialog exchanges. It views the entire exchange in the Automated Planning (AP) paradigm, representing the dialog sequences (speech acts) in the form of action sequences that modify the state of the world upon execution, gradually propelling the state to a desired goal. We then exploit intra-action and inter-action dependencies, encoding them in the form of constraints. We attempt to satisfy these constraints using aweighted maximum satisfiability model known as MAX-SAT, and convert the solution into a speech model. This model could have many uses, such as planning of fresh dialogs. In this study, the learnt model is used to predict speech acts in the dialog sequences using the sequence labeling (predicting future acts based on previously seen ones) capabilities of the LSTM (Long Short Term Memory) class of recurrent neural networks. Encouraging empirical results demonstrate the utility of this learnt model and its long term potential to facilitate autonomous behavioral planning of robots, an aspect to be explored in future works.

scholarly journals An LSTM-Based Method with Attention Mechanism for Travel Time Prediction

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 861 ◽  
Author(s):  
Xiangdong Ran ◽  
Zhiguang Shan ◽  
Yufei Fang ◽  
Chuang Lin
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Traffic Prediction ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Departure Time ◽  
Long Short Term Memory

Traffic prediction is based on modeling the complex non-linear spatiotemporal traffic dynamics in road network. In recent years, Long Short-Term Memory has been applied to traffic prediction, achieving better performance. The existing Long Short-Term Memory methods for traffic prediction have two drawbacks: they do not use the departure time through the links for traffic prediction, and the way of modeling long-term dependence in time series is not direct in terms of traffic prediction. Attention mechanism is implemented by constructing a neural network according to its task and has recently demonstrated success in a wide range of tasks. In this paper, we propose an Long Short-Term Memory-based method with attention mechanism for travel time prediction. We present the proposed model in a tree structure. The proposed model substitutes a tree structure with attention mechanism for the unfold way of standard Long Short-Term Memory to construct the depth of Long Short-Term Memory and modeling long-term dependence. The attention mechanism is over the output layer of each Long Short-Term Memory unit. The departure time is used as the aspect of the attention mechanism and the attention mechanism integrates departure time into the proposed model. We use AdaGrad method for training the proposed model. Based on the datasets provided by Highways England, the experimental results show that the proposed model can achieve better accuracy than the Long Short-Term Memory and other baseline methods. The case study suggests that the departure time is effectively employed by using attention mechanism.

scholarly journals Long Short-Term Memory with Dynamic Skip Connections

2019 ◽  
Vol 33 ◽  
pp. 6481-6488 ◽  
Author(s):  
Tao Gui ◽  
Qi Zhang ◽  
Lujun Zhao ◽  
Yaosong Lin ◽  
Minlong Peng ◽  
...  
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Training Data ◽  
Sequential Data ◽  
Short Term ◽  
Term Memory ◽  
Transition Functions ◽  
Proposed Model ◽  
Long Short Term Memory

In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length. However, LSTM can still experience difficulty in capturing long-term dependencies. In this work, we tried to alleviate this problem by introducing a dynamic skip connection, which can learn to directly connect two dependent words. Since there is no dependency information in the training data, we propose a novel reinforcement learning-based method to model the dependency relationship and connect dependent words. The proposed model computes the recurrent transition functions based on the skip connections, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially. Our experimental results on three natural language processing tasks demonstrate that the proposed method can achieve better performance than existing methods. In the number prediction experiment, the proposed model outperformed LSTM with respect to accuracy by nearly 20%.

scholarly journals Biomedical named entity recognition using deep neural networks with contextual information

2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Hyejin Cho ◽  
Hyunju Lee
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Contextual Information ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Learning Approaches ◽  
Short Term ◽  
Term Memory ◽  
Named Entity ◽  
Long Short Term Memory

Abstract Background In biomedical text mining, named entity recognition (NER) is an important task used to extract information from biomedical articles. Previously proposed methods for NER are dictionary- or rule-based methods and machine learning approaches. However, these traditional approaches are heavily reliant on large-scale dictionaries, target-specific rules, or well-constructed corpora. These methods to NER have been superseded by the deep learning-based approach that is independent of hand-crafted features. However, although such methods of NER employ additional conditional random fields (CRF) to capture important correlations between neighboring labels, they often do not incorporate all the contextual information from text into the deep learning layers. Results We propose herein an NER system for biomedical entities by incorporating n-grams with bi-directional long short-term memory (BiLSTM) and CRF; this system is referred to as a contextual long short-term memory networks with CRF (CLSTM). We assess the CLSTM model on three corpora: the disease corpus of the National Center for Biotechnology Information (NCBI), the BioCreative II Gene Mention corpus (GM), and the BioCreative V Chemical Disease Relation corpus (CDR). Our framework was compared with several deep learning approaches, such as BiLSTM, BiLSTM with CRF, GRAM-CNN, and BERT. On the NCBI corpus, our model recorded an F-score of 85.68% for the NER of diseases, showing an improvement of 1.50% over previous methods. Moreover, although BERT used transfer learning by incorporating more than 2.5 billion words, our system showed similar performance with BERT with an F-scores of 81.44% for gene NER on the GM corpus and a outperformed F-score of 86.44% for the NER of chemicals and diseases on the CDR corpus. We conclude that our method significantly improves performance on biomedical NER tasks. Conclusion The proposed approach is robust in recognizing biological entities in text.

Sign in / Sign up

Export Citation Format

Share Document