scholarly journals Time Series Forecasting and Classification Models Based on Recurrent with Attention Mechanism and Generative Adversarial Networks

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7211
Author(s):  
Kun Zhou ◽  
Wenyong Wang ◽  
Teng Hu ◽  
Kai Deng
Keyword(s):  
Time Series ◽  
Short Term Memory ◽  
Time Series Classification ◽  
Training Process ◽  
Short Term ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Term Memory ◽  
Adversarial Network ◽  
Long Short Term Memory

Time series classification and forecasting have long been studied with the traditional statistical methods. Recently, deep learning achieved remarkable successes in areas such as image, text, video, audio processing, etc. However, research studies conducted with deep neural networks in these fields are not abundant. Therefore, in this paper, we aim to propose and evaluate several state-of-the-art neural network models in these fields. We first review the basics of representative models, namely long short-term memory and its variants, the temporal convolutional network and the generative adversarial network. Then, long short-term memory with autoencoder and attention-based models, the temporal convolutional network and the generative adversarial model are proposed and applied to time series classification and forecasting. Gaussian sliding window weights are proposed to speed the training process up. Finally, the performances of the proposed methods are assessed using five optimizers and loss functions with the public benchmark datasets, and comparisons between the proposed temporal convolutional network and several classical models are conducted. Experiments show the proposed models’ effectiveness and confirm that the temporal convolutional network is superior to long short-term memory models in sequence modeling. We conclude that the proposed temporal convolutional network reduces time consumption to around 80% compared to others while retaining the same accuracy. The unstable training process for generative adversarial network is circumvented by tuning hyperparameters and carefully choosing the appropriate optimizer of “Adam”. The proposed generative adversarial network also achieves comparable forecasting accuracy with traditional methods.

scholarly journals A generative adversarial network–based method for generating negative financial samples

2020 ◽  
Vol 16 (2) ◽  
pp. 155014772090705
Author(s):  
Zhaohui Zhang ◽  
Lijun Yang ◽  
Ligong Chen ◽  
Qiuwen Liu ◽  
Ying Meng ◽  
...  
Keyword(s):  
Evaluation Method ◽  
Short Term Memory ◽  
Data Distribution ◽  
Real Data ◽  
Generation Model ◽  
Short Term ◽  
Generative Adversarial Network ◽  
Term Memory ◽  
Adversarial Network ◽  
Long Short Term Memory

In financial anti-fraud field, negative samples are small and sparse with serious sample imbalanced problem. Generating negative samples consistent with original data to naturally solve imbalanced problem is a serious problem. This article proposes a new method to solve this problem. We introduce a new generation model, combined Generative Adversarial Network with Long Short-Term Memory network for one-dimensional negative financial samples. The characteristic association between transaction sequences can be learned by long short-term memory layer, and the generator covers real data distribution by the adversarial discriminator with time-sequence. Mapping data distribution to feature space is a common evaluation method of synthetic data; however, relationships between data attributes have been ignored in online transactions. We define a comprehensive evaluation method to evaluate the validity of generated samples from data distribution and attribute characteristics. Experimental results on real bank B2B transaction data show that the proposed model has higher overall ratings, which is 10% higher than traditional generation models. Finally, well-trained model is used to generate negative samples and form new dataset. The classification results on new datasets show that precision and recall are all higher than baseline models. Our work has a certain practical value and provides a new idea to solve imbalanced problem in whatever fields.

scholarly journals Conditional LSTM-GAN for Melody Generation from Lyrics

2021 ◽  
Vol 17 (1) ◽  
pp. 1-20
Author(s):  
Yi Yu ◽  
Abhishek Srivastava ◽  
Simon Canales
Keyword(s):  
Short Term Memory ◽  
Generative Model ◽  
Research Progress ◽  
Short Term ◽  
Large Dataset ◽  
Generative Adversarial Network ◽  
Term Memory ◽  
Adversarial Network ◽  
Challenging Research ◽  
Long Short Term Memory

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limited availability of a paired lyrics–melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory (LSTM)–Generative Adversarial Network for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of predicted melody are generated simultaneously. Extensive experimental results have proved the effectiveness of our proposed lyrics-to-melody generative model, where plausible and tuneful sequences can be inferred from lyrics.

Prediction of InSAR deformation time-series using a long short-term memory neural network

2021 ◽  
Vol 42 (18) ◽  
pp. 6921-6944
Author(s):  
Yi Chen ◽  
Yi He ◽  
Lifeng Zhang ◽  
Youdong Chen ◽  
Hongyu Pu ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Prognostics and RUL Estimations of SAC305, SAC105 and SnAg Solders Under Temperature and Vibration Using Long Short-Term Memory (LSTM) Deep Learning

2021 ◽  
Author(s):  
Pradeep Lall ◽  
Tony Thomas ◽  
Ken Blecker
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Remaining Useful Life ◽  
Operating Conditions ◽  
Short Term ◽  
Term Memory ◽  
Rectangular Pattern ◽  
Useful Life ◽  
Long Short Term Memory

Abstract Prognostics and Remaining Useful Life (RUL) estimations of complex systems are essential to operational safety, increased efficiency, and help to schedule maintenance proactively. Modeling the remaining useful life of a system with many complexities is possible with the rapid development in the field of deep learning as a computational technique for failure prediction. Deep learning can adapt to multivariate parameters complex and nonlinear behavior, which is difficult using traditional time-series models for forecasting and prediction purposes. In this paper, a deep learning approach based on Long Short-Term Memory (LSTM) network is used to predict the remaining useful life of the PCB at different conditions of temperature and vibration. This technique can identify the different underlying patterns in the time series that can predict the RUL. This study involves feature vector identification and RUL estimations for SAC305, SAC105, and Tin Lead solder PCBs under different vibration levels and temperature conditions. The acceleration levels of vibration are fixed at 5g and 10g, while the temperature levels are 55°C and 100°C. The test board is a multilayer FR4 configuration with JEDEC standard dimensions consists of twelve packages arranged in a rectangular pattern. Strain signals are acquired from the backside of the PCB at symmetric locations to identify the failure of all the packages during vibration. The strain signals are resistance values that are acquired simultaneously during the experiment until the failure of most of the packages on the board. The feature vectors are identified from statistical analysis on the strain signals frequency and instantaneous frequency components. The principal component analysis is used as a data reduction technique to identify the different patterns produced from the four strain signals with failures of the packages during vibration. LSTM deep learning method is used to model the RUL of the packages at different individual operating conditions of vibration for all three solder materials involved in this study. A combined model for RUL prediction for a material that can take care of the changes in the operating conditions is also modeled for each material.

Sign in / Sign up

Export Citation Format

Share Document