A Novel Vehicle Destination Prediction Model With Expandable Features Using Attention Mechanism and Variational Autoencoder

2022 ◽  
pp. 1-10
Author(s):  
Xiangyang Wu ◽  
Weite Zhu ◽  
Zhen Liu ◽  
Zhen Zhang
Keyword(s):  
Prediction Model ◽  
Attention Mechanism ◽  
Variational Autoencoder

scholarly journals Research on Short-Term Load Prediction Based on Seq2seq Model

Energies ◽  
2019 ◽  
Vol 12 (16) ◽  
pp. 3199 ◽  
Author(s):  
Gangjun Gong ◽  
Xiaonan An ◽  
Nawaraj Kumar Mahato ◽  
Shuyan Sun ◽  
Si Chen ◽  
...  
Keyword(s):  
Power System ◽  
Prediction Model ◽  
Short Term Memory ◽  
Predictive Performance ◽  
Weather Conditions ◽  
Attention Mechanism ◽  
Load Prediction ◽  
Short Term ◽  
System Load ◽  
The Impact

Electricity load prediction is the primary basis on which power-related departments to make logical and effective generation plans and scientific scheduling plans for the most effective power utilization. The perpetual evolution of deep learning has recommended advanced and innovative concepts for short-term load prediction. Taking into consideration the time and nonlinear characteristics of power system load data and further considering the impact of historical and future information on the current state, this paper proposes a Seq2seq short-term load prediction model based on a long short-term memory network (LSTM). Firstly, the periodic fluctuation characteristics of users’ load data are analyzed, establishing a correlation of the load data so as to determine the model’s order in the time series. Secondly, the specifications of the Seq2seq model are given preference and a coalescence of the Residual mechanism (Residual) and the two Attention mechanisms (Attention) is developed. Then, comparing the predictive performance of the model under different types of Attention mechanism, this paper finally adopts the Seq2seq short-term load prediction model of Residual LSTM and the Bahdanau Attention mechanism. Eventually, the prediction model obtains better results when merging the actual power system load data of a certain place. In order to validate the developed model, the Seq2seq was compared with recurrent neural network (RNN), LSTM, and gated recurrent unit (GRU) algorithms. Last but not least, the performance indices were calculated. when training and testing the model with power system load data, it was noted that the root mean square error (RMSE) of Seq2seq was decreased by 6.61%, 16.95%, and 7.80% compared with RNN, LSTM, and GRU, respectively. In addition, a supplementary case study was carried out using data for a small power system considering different weather conditions and user behaviors in order to confirm the applicability and stability of the proposed model. The Seq2seq model for short-term load prediction can be reported to demonstrate superiority in all areas, exhibiting better prediction and stable performance.

scholarly journals TEP2MP: A text-emotion prediction model oriented to multi-participant text-conversation scenario with hybrid attention enhancement

2022 ◽  
Vol 19 (3) ◽  
pp. 2671-2699
Author(s):  
Huan Rong ◽  
◽  
Tinghuai Ma ◽  
Xinyu Cao ◽  
Xin Yu ◽  
...  
Keyword(s):  
Prediction Model ◽  
Online Social Networks ◽  
Rapid Development ◽  
Attention Mechanism ◽  
Text Representation ◽  
Search Mechanism ◽  
Proposed Model ◽  
The Government ◽  
Prediction Efficiency ◽  
Target Speaker

<abstract> <p>With the rapid development of online social networks, text-communication has become an indispensable part of daily life. Mining the emotion hidden behind the conversation-text is of prime significance and application value when it comes to the government public-opinion supervision, enterprise decision-making, etc. Therefore, in this paper, we propose a text emotion prediction model in a multi-participant text-conversation scenario, which aims to effectively predict the emotion of the text to be posted by target speaker in the future. Specifically, first, an <italic>affective space mapping</italic> is constructed, which represents the original conversation-text as an n-dimensional <italic>affective vector</italic> so as to obtain the text representation on different emotion categories. Second, a similar scene search mechanism is adopted to seek several sub-sequences which contain similar tendency on emotion shift to that of the current conversation scene. Finally, the text emotion prediction model is constructed in a two-layer encoder-decoder structure with the emotion fusion and hybrid attention mechanism introduced at the encoder and decoder side respectively. According to the experimental results, our proposed model can achieve an overall best performance on emotion prediction due to the auxiliary features extracted from similar scenes and the adoption of emotion fusion as well as the hybrid attention mechanism. At the same time, the prediction efficiency can still be controlled at an acceptable level.</p> </abstract>

scholarly journals Learning to balance the coherence and diversity of response generation in generation-based chatbots

2020 ◽  
Vol 17 (4) ◽  
pp. 172988142095300
Author(s):  
Shuliang Wang ◽  
Dapeng Li ◽  
Jing Geng ◽  
Longxing Yang ◽  
Hongyong Leng
Keyword(s):  
Attention Mechanism ◽  
Experimental Results ◽  
Generation Model ◽  
Improved Method ◽  
Variational Autoencoder

Generating response with both coherence and diversity is a challenging task in generation-based chatbots. It is more difficult to improve the coherence and diversity of dialog generation at the same time in the response generation model. In this article, we propose an improved method that improves the coherence and diversity of dialog generation by changing the model to use gamma sampling and adding attention mechanism to the knowledge-guided conditional variational autoencoder. The experimental results demonstrate that our proposed method can significantly improve the coherence and diversity of knowledge-guided conditional variational autoencoder for response generation in generation-based chatbots at the same time.

Product click-through rate prediction model integrating self-attention mechanism

2021 ◽  
Author(s):  
Tong Zhu ◽  
Shuqin Li ◽  
Chunquan Liang ◽  
Bin Liu ◽  
Xiaopeng Li
Keyword(s):  
Prediction Model ◽  
Attention Mechanism ◽  
Rate Prediction ◽  
Click Through Rate

Intelligent fault prediction of rolling bearing based on gate recurrent unit and hybrid autoencoder

2020 ◽  
pp. 095440622094103
Author(s):  
Changchang Che ◽  
Huawei Wang ◽  
Qiang Fu ◽  
Xiaomei Ni
Keyword(s):  
Time Series ◽  
Prediction Model ◽  
Data Augmentation ◽  
Time Series Prediction ◽  
Vibration Signal ◽  
Rolling Bearing ◽  
Fault Prediction ◽  
Denoising Autoencoder ◽  
Operation Condition ◽  
Variational Autoencoder

Accurate fault prediction of rolling bearing can predict the operation condition in advance, which is an important means to ensure the safety and reliability of rotating machinery. Aimed at the data processing of rolling bearing vibration signal with multi-fault and long time series, an intelligent fault prediction model based on gate recurrent unit and hybrid autoencoder is proposed in this paper. Firstly, vibration signals of multi-faults are brought into multi-layer gate recurrent unit model for multi-step and multi-variable time series prediction. Secondly, variational autoencoder is used for data augmentation of fault samples. Thirdly, the augmented fault samples are brought into stacked denoising autoencoder for noise reduction and fault prediction. Finally, fault prediction results of rolling bearing can be achieved on the basis of gate recurrent unit and hybrid autoencoder of variational autoencoder and stacked denoising autoencoder. The bearing datasets of Case Western Reserve University are used to verify the effectiveness of the proposed method. Comparative experiment results show that the proposed fault prediction model has more accurate time series prediction result and higher fault prediction accuracy than other deep learning models. With 98.68% accuracy of fault prediction, the proposed fault prediction model can be taken as an effective tool for intelligent predictive maintenance of rolling bearing.

scholarly journals NTAM-LSTM models of network traffic prediction

2022 ◽  
Vol 355 ◽  
pp. 02007
Author(s):  
Jihong Zhao ◽  
Xiaoyuan He
Keyword(s):  
Prediction Model ◽  
Network Traffic ◽  
Prediction Models ◽  
Rapid Development ◽  
Attention Mechanism ◽  
Accurate Prediction ◽  
Traffic Prediction ◽  
Network Resources ◽  
Multiple Characteristics ◽  
Lstm Network

Accurate prediction of network traffic is very important in allocating network resources. With the rapid development of network technology, network traffic becomes more complex and diverse. The traditional network traffic prediction model cannot accurately predict the current network traffic within the effective time. This paper proposes a Network Traffic Prediction Model----NTAM-LSTM, which based on Attention Mechanism with Long and Short Time Memory. Firstly, the model preprocesses the historical dataset of network traffic with multiple characteristics. Then the LSTM network is used to make initial prediction for the processed dataset. Finally, attention mechanism is introduced to get more accurate prediction results. Compared with other network traffic prediction models, NTAM-LSTM prediction model can achieve higher prediction accuracy and take shorter running time.

Sign in / Sign up

Export Citation Format

Share Document