scholarly journals What to Do Next: Modeling User Behaviors by Time-LSTM

2017 ◽  
Author(s):  
Yu Zhu ◽  
Hao Li ◽  
Yikang Liao ◽  
Beidou Wang ◽  
Ziyu Guan ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Language Modeling ◽  
Sequential Data ◽  
Sequential Order ◽  
Short Term ◽  
Time Intervals ◽  
User Behaviors ◽  
Real World Datasets

Recently, Recurrent Neural Network (RNN) solutions for recommender systems (RS) are becoming increasingly popular. The insight is that, there exist some intrinsic patterns in the sequence of users' actions, and RNN has been proved to perform excellently when modeling sequential data. In traditional tasks such as language modeling, RNN solutions usually only consider the sequential order of objects without the notion of interval. However, in RS, time intervals between users' actions are of significant importance in capturing the relations of users' actions and the traditional RNN architectures are not good at modeling them. In this paper, we propose a new LSTM variant, i.e. Time-LSTM, to model users' sequential actions. Time-LSTM equips LSTM with time gates to model time intervals. These time gates are specifically designed, so that compared to the traditional RNN solutions, Time-LSTM better captures both of users' short-term and long-term interests, so as to improve the recommendation performance. Experimental results on two real-world datasets show the superiority of the recommendation method using Time-LSTM over the traditional methods.

scholarly journals Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Jianlei Zhang ◽  
Yukun Zeng ◽  
Binil Starly
Keyword(s):  
Neural Network ◽  
Tool Wear ◽  
Machine Tool ◽  
Recurrent Neural Network ◽  
Machine Tools ◽  
Prediction Performance ◽  
Sequential Data ◽  
Diagnosis And Prognosis ◽  
Proposed Model

AbstractData-driven approaches for machine tool wear diagnosis and prognosis are gaining attention in the past few years. The goal of our study is to advance the adaptability, flexibility, prediction performance, and prediction horizon for online monitoring and prediction. This paper proposes the use of a recent deep learning method, based on Gated Recurrent Neural Network architecture, including Long Short Term Memory (LSTM), which try to captures long-term dependencies than regular Recurrent Neural Network method for modeling sequential data, and also the mechanism to realize the online diagnosis and prognosis and remaining useful life (RUL) prediction with indirect measurement collected during the manufacturing process. Existing models are usually tool-specific and can hardly be generalized to other scenarios such as for different tools or operating environments. Different from current methods, the proposed model requires no prior knowledge about the system and thus can be generalized to different scenarios and machine tools. With inherent memory units, the proposed model can also capture long-term dependencies while learning from sequential data such as those collected by condition monitoring sensors, which means it can be accommodated to machine tools with varying life and increase the prediction performance. To prove the validity of the proposed approach, we conducted multiple experiments on a milling machine cutting tool and applied the model for online diagnosis and RUL prediction. Without loss of generality, we incorporate a system transition function and system observation function into the neural net and trained it with signal data from a minimally intrusive vibration sensor. The experiment results showed that our LSTM-based model achieved the best overall accuracy among other methods, with a minimal Mean Square Error (MSE) for tool wear prediction and RUL prediction respectively.

scholarly journals Long Short Term Memory Recurrent Network for Standard and Poor’s 500 Index Modelling

2018 ◽  
Vol 7 (4.15) ◽  
pp. 25 ◽  
Author(s):  
Said Jadid Abdulkadir ◽  
Hitham Alhussian ◽  
Muhammad Nazmi ◽  
Asim A Elsheikh
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Forecasting time-series data are imperative especially when planning is required through modelling using uncertain knowledge of future events. Recurrent neural network models have been applied in the industry and outperform standard artificial neural networks in forecasting, but fail in long term time-series forecasting due to the vanishing gradient problem. This study offers a robust solution that can be implemented for long-term forecasting using a special architecture of recurrent neural network known as Long Short Term Memory (LSTM) model to overcome the vanishing gradient problem. LSTM is specially designed to avoid the long-term dependency problem as their default behavior. Empirical analysis is performed using quantitative forecasting metrics and comparative model performance on the forecasted outputs. An evaluation analysis is performed to validate that the LSTM model provides better forecasted outputs on Standard & Poor’s 500 Index (S&P 500) in terms of error metrics as compared to other forecasting models.  

scholarly journals Context-Aware Short-Term Interest First Model for Session-Based Recommendation

2021 ◽  
Author(s):  
Haomei Duan ◽  
Jinghua Zhu
Keyword(s):  
Neural Network ◽  
General Interest ◽  
Long Term Memory ◽  
Graph Structure ◽  
Context Aware ◽  
User Profiles ◽  
Short Term ◽  
Term Memory ◽  
Real World Datasets

In the case that user profiles are not available, the recommendation based on anonymous session is particularly important, which aims to predict the items that the user may click at the next moment based on the user's access sequence over a while. In recent years, with the development of recurrent neural network, attention mechanism, and graph neural network, the performance of session-based recommendation has been greatly improved. However, the previous methods did not comprehensively consider the context dependencies and short-term interest first of the session. Therefore, we propose a context-aware short-term interest first model (CASIF).The aim of this paper is improve the accuracy of recommendations by combining context and short-term interest. In CASIF, we dynamically construct a graph structure for session sequences and capture rich context dependencies via graph neural network (GNN), latent feature vectors are captured as inputs of the next step. Then we build the shortterm interest first module, which can to capture the user's general interest from the session in the context of long-term memory, at the same time get the user's current interest from the item of the last click. In the end, the short-term and long-term interest are combined as the final interest and multiplied by the candidate vector to obtain the recommendation probability. Finally, a large number of experiments on two real-world datasets demonstrate the effectiveness of our proposed method.

scholarly journals Memory Augmented Graph Neural Networks for Sequential Recommendation

2020 ◽  
Vol 34 (04) ◽  
pp. 5045-5052 ◽  
Author(s):  
Chen Ma ◽  
Liheng Ma ◽  
Yingxue Zhang ◽  
Jianing Sun ◽  
Xue Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Recommender Systems ◽  
Performance Metrics ◽  
Contextual Information ◽  
Short Term ◽  
User Interests ◽  
User Behaviors ◽  
Real World Datasets ◽  
Graph Neural Networks ◽  
Occurrence Patterns

The chronological order of user-item interactions can reveal time-evolving and sequential user behaviors in many recommender systems. The items that users will interact with may depend on the items accessed in the past. However, the substantial increase of users and items makes sequential recommender systems still face non-trivial challenges: (1) the hardness of modeling the short-term user interests; (2) the difficulty of capturing the long-term user interests; (3) the effective modeling of item co-occurrence patterns. To tackle these challenges, we propose a memory augmented graph neural network (MA-GNN) to capture both the long- and short-term user interests. Specifically, we apply a graph neural network to model the item contextual information within a short-term period and utilize a shared memory network to capture the long-range dependencies between items. In addition to the modeling of user interests, we employ a bilinear function to capture the co-occurrence patterns of related items. We extensively evaluate our model on five real-world datasets, comparing with several state-of-the-art methods and using a variety of performance metrics. The experimental results demonstrate the effectiveness of our model for the task of Top-K sequential recommendation.

scholarly journals Multiplexed neurochemical transmission emulated using a dual-excitatory synaptic transistor

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Mingxue Ma ◽  
Yao Ni ◽  
Zirong Chi ◽  
Wanqing Meng ◽  
Haiyang Yu ◽  
...  
Keyword(s):  
Neural Network ◽  
Central Nervous System ◽  
Nervous System ◽  
Long Term Potentiation ◽  
Short Term ◽  
Term Potentiation ◽  
Binary Neural Network ◽  
Neuromorphic Systems ◽  
Human Brains

AbstractThe ability to emulate multiplexed neurochemical transmission is an important step toward mimicking complex brain activities. Glutamate and dopamine are neurotransmitters that regulate thinking and impulse signals independently or synergistically. However, emulation of such simultaneous neurotransmission is still challenging. Here we report design and fabrication of synaptic transistor that emulates multiplexed neurochemical transmission of glutamate and dopamine. The device can perform glutamate-induced long-term potentiation, dopamine-induced short-term potentiation, or co-release-induced depression under particular stimulus patterns. More importantly, a balanced ternary system that uses our ambipolar synaptic device backtrack input ‘true’, ‘false’ and ‘unknown’ logic signals; this process is more similar to the information processing in human brains than a traditional binary neural network. This work provides new insight for neuromorphic systems to establish new principles to reproduce the complexity of a mammalian central nervous system from simple basic units.

Sign in / Sign up

Export Citation Format

Share Document