scholarly journals Pay attention and you won’t lose it: a deep learning approach to sequence imputation

2019 ◽  
Vol 5 ◽  
pp. e210
Author(s):  
Ilia Sucholutsky ◽  
Apurva Narayan ◽  
Matthias Schonlau ◽  
Sebastian Fischmeister
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Learning Approach ◽  
Short Term ◽  
Fold Reduction ◽  
Data Restoration ◽  
Major Bottleneck ◽  
Long Short Term Memory ◽  
End Stage

In most areas of machine learning, it is assumed that data quality is fairly consistent between training and inference. Unfortunately, in real systems, data are plagued by noise, loss, and various other quality reducing factors. While a number of deep learning algorithms solve end-stage problems of prediction and classification, very few aim to solve the intermediate problems of data pre-processing, cleaning, and restoration. Long Short-Term Memory (LSTM) networks have previously been proposed as a solution for data restoration, but they suffer from a major bottleneck: a large number of sequential operations. We propose using attention mechanisms to entirely replace the recurrent components of these data-restoration networks. We demonstrate that such an approach leads to reduced model sizes by as many as two orders of magnitude, a 2-fold to 4-fold reduction in training times, and 95% accuracy for automotive data restoration. We also show in a case study that this approach improves the performance of downstream algorithms reliant on clean data.

scholarly journals Electrocorticography based motor imagery movements classification using long short-term memory (LSTM) based on deep learning approach

2020 ◽  
Vol 2 (2) ◽  
Author(s):  
Mamunur Rashid ◽  
Minarul Islam ◽  
Norizam Sulaiman ◽  
Bifta Sama Bari ◽  
Ripon Kumar Saha ◽  
...  
Keyword(s):  
Deep Learning ◽  
Motor Imagery ◽  
Short Term Memory ◽  
Learning Approach ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

scholarly journals A Deep Learning Approach for the Romanized Tunisian Dialect Identification

2020 ◽  
Vol 17 (6) ◽  
pp. 935-946
Author(s):  
Jihene Younes ◽  
Hadhemi Achour ◽  
Emna Souissi ◽  
Ahmed Ferchichi
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Conditional Random Fields ◽  
Short Term Memory ◽  
Learning Approach ◽  
Short Term ◽  
The Social ◽  
Long Short Term Memory ◽  
N Gram ◽  
Textual Content

Language identification is an important task in natural language processing that consists in determining the language of a given text. It has increasingly picked the interest of researchers for the past few years, especially for code-switching informal textual content. In this paper, we focus on the identification of the Romanized user-generated Tunisian dialect on the social web. We segment and annotate a corpus extracted from social media and propose a deep learning approach for the identification task. We use a Bidirectional Long Short-Term Memory neural network with Conditional Random Fields decoding (BLSTM-CRF). For word embeddings, we combine word-character BLSTM vector representation and Fast Text embeddings that takes into consideration character n-gram features. The overall accuracy obtained is 98.65%.

scholarly journals Deep Learning Approach for Estimating Workability of Self-Compacting Concrete from Mixing Image Sequences

2018 ◽  
Vol 2018 ◽  
pp. 1-16 ◽  
Author(s):  
Zhongcong Ding ◽  
Xuehui An
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Image Sequences ◽  
Learning Approach ◽  
Self Compacting Concrete ◽  
Short Term ◽  
Slump Flow ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Funnel Flow

We propose a deep learning approach to better utilize the spatial and temporal information obtained from image sequences of the self-compacting concrete- (SCC-) mixing process to recover SCC characteristics in terms of the predicted slump flow value (SF) and V-funnel flow time (VF). The proposed model integrates features of the convolutional neural network and long short-term memory and is trained to extract features and compute an estimate. The performance of the method is evaluated using the testing set. The results indicate that the proposed method could potentially be used to automatically estimate SCC workability.

scholarly journals Prediction of Covid-19 Daily Case in Indonesia Using Long Short Term Memory Method

Teknika ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 62-67
Author(s):  
Faisal Dharma Adhinata ◽  
Diovianto Putra Rakhmadani
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Short Term Memory ◽  
Machine Learning Techniques ◽  
Learning Approach ◽  
Short Term ◽  
Term Memory ◽  
Learning Techniques ◽  
Long Short Term Memory ◽  
The Impact

The impact of this pandemic affects various sectors in Indonesia, especially in the economic sector, due to the large-scale social restrictions policy to suppress this case's growth. The details of the growth of Covid-19 in Indonesia are still fluctuating and cannot be fully understood. Recently it has been developed by researchers related to the prediction of Covid-19 cases in various countries. One of them is using a machine learning technique approach to predict cases of daily increase Covid-19. However, the use of machine learning techniques results in the MSE error value in the thousands. This high number indicates that the prediction data using the model is still a high error rate compared to the actual data. In this study, we propose a deep learning approach using the Long Short Term Memory (LSTM) method to build a prediction model for the daily increase cases of Covid-19. This study's LSTM model architecture uses the LSTM layer, Dropout layer, Dense, and Linear Activation Function. Based on various hyperparameter experiments, using the number of neurons 10, batch size 32, and epochs 50, the MSE values were 0.0308, RMSE 0.1758, and MAE 0.13. These results prove that the deep learning approach produces a smaller error value than machine learning techniques, even closer to zero.

scholarly journals Application of Rough and Fuzzy Set Theory for Prediction of Stochastic Wind Speed Data Using Long Short-Term Memory

Atmosphere ◽  
2021 ◽  
Vol 12 (7) ◽  
pp. 924
Author(s):  
Moslem Imani ◽  
Hoda Fakour ◽  
Wen-Hau Lan ◽  
Huan-Chin Kao ◽  
Chi Ming Lee ◽  
...  
Keyword(s):  
Deep Learning ◽  
Wind Speed ◽  
Set Theory ◽  
Rough Set ◽  
Fuzzy Set Theory ◽  
Short Term Memory ◽  
Learning Model ◽  
Short Term ◽  
Long Short Term Memory ◽  
Deep Learning Model

Despite the great significance of precisely forecasting the wind speed for development of the new and clean energy technology and stable grid operators, the stochasticity of wind speed makes the prediction a complex and challenging task. For improving the security and economic performance of power grids, accurate short-term wind power forecasting is crucial. In this paper, a deep learning model (Long Short-term Memory (LSTM)) has been proposed for wind speed prediction. Knowing that wind speed time series is nonlinear stochastic, the mutual information (MI) approach was used to find the best subset from the data by maximizing the joint MI between subset and target output. To enhance the accuracy and reduce input characteristics and data uncertainties, rough set and interval type-2 fuzzy set theory are combined in the proposed deep learning model. Wind speed data from an international airport station in the southern coast of Iran Bandar-Abbas City was used as the original input dataset for the optimized deep learning model. Based on the statistical results, the rough set LSTM (RST-LSTM) model showed better prediction accuracy than fuzzy and original LSTM, as well as traditional neural networks, with the lowest error for training and testing datasets in different time horizons. The suggested model can support the optimization of the control approach and the smooth procedure of power system. The results confirm the superior capabilities of deep learning techniques for wind speed forecasting, which could also inspire new applications in meteorology assessment.

Sign in / Sign up

Export Citation Format

Share Document