Generative Text Steganography Based on LSTM Network and Attention Mechanism with Keywords

The widespread use of text over online social networks makes it quite suitable for steganography. Conventional text steganography usually transmits the secret data by either slightly modifying the given text or hiding the secret data through synonym replacement. The rapid development of deep neural networks (DNNs) has led automatically generating the steganographic text to become an important and promising topic. This has motivated us to propose a novel generative text steganographic method based on long short-term memory (LSTM) network in this paper. We apply attention mechanism with keywords to the LSTM network to generate the steganographic text. Experiments show that, compared to the related work, the steganographic text generated by the proposed method is of higher semantic quality and more capable of resisting against steganalysis, which has shown the superiority.

Download Full-text

Application of Heartbeat-Attention Mechanism for Detection of Myocardial Infarction Using 12-Lead ECG Records

Applied Sciences ◽

10.3390/app9163328 ◽

2019 ◽

Vol 9 (16) ◽

pp. 3328 ◽

Cited By ~ 2

Author(s):

Zhang ◽

Keyword(s):

Myocardial Infarction ◽

Short Term Memory ◽

Attention Mechanism ◽

Short Term ◽

Sliding Windows ◽

Long Short Term Memory ◽

The Difference ◽

Lstm Network ◽

Electrocardiogram Ecg ◽

Ecg Database

Early detection and effective treatment of myocardial infarction can prevent the deterioration of ischemic heart disease and greatly reduce the possibility of sudden death. On the basis of standard 12-lead electrocardiogram (ECG) records, this paper proposes a bidirectional, long short-term memory (Bi-LSTM) network with a heartbeat-attention mechanism to effectively and automatically detect myocardial infarction (MI). First, we divide the standard 12-lead ECG records into sliding windows with the same number of heartbeats. Subsequently, we do not use any labels of heartbeats to train the Bi-LSTM network and the heartbeat-attention mechanism is applied to automatically weight the difference between unlabeled heartbeats. Finally, our method is validated by patients’ complete ECG records and real labels in the Physikalisch-Technische Bundesanstalt (PTB) diagnostic ECG database. When compared with the same network without the heartbeat-attention mechanism or other existing methods, our method achieves comparable or better performance. The accuracy, sensitivity, and specificity reach 94.77%, 95.58%, and 90.48%, respectively.

Download Full-text

Dam Deformation Interpretation and Prediction Based on a Long Short-Term Memory Model Coupled with an Attention Mechanism

Applied Sciences ◽

10.3390/app11146625 ◽

2021 ◽

Vol 11 (14) ◽

pp. 6625

Author(s):

Yan Su ◽

Kailiang Weng ◽

Chuan Lin ◽

Zeqin Chen

Keyword(s):

Short Term Memory ◽

Attention Mechanism ◽

Impact Factors ◽

Nonlinear Prediction ◽

Time Dimension ◽

Short Term ◽

Deformation Prediction ◽

Term Memory ◽

Long Short Term Memory ◽

Dam Deformation

An accurate dam deformation prediction model is vital to a dam safety monitoring system, as it helps assess and manage dam risks. Most traditional dam deformation prediction algorithms ignore the interpretation and evaluation of variables and lack qualitative measures. This paper proposes a data processing framework that uses a long short-term memory (LSTM) model coupled with an attention mechanism to predict the deformation response of a dam structure. First, the random forest (RF) model is introduced to assess the relative importance of impact factors and screen input variables. Secondly, the density-based spatial clustering of applications with noise (DBSCAN) method is used to identify and filter the equipment based abnormal values to reduce the random error in the measurements. Finally, the coupled model is used to focus on important factors in the time dimension in order to obtain more accurate nonlinear prediction results. The results of the case study show that, of all tested methods, the proposed coupled method performed best. In addition, it was found that temperature and water level both have significant impacts on dam deformation and can serve as reliable metrics for dam management.

Download Full-text

Abnormal Detection of Electricity Consumption of User Based on Particle Swarm Optimization and Long Short Term Memory With the Attention Mechanism

IEEE Access ◽

10.1109/access.2021.3062675 ◽

2021 ◽

Vol 9 ◽

pp. 47252-47265

Author(s):

Jiahao Bian ◽

Lei Wang ◽

Rafal Scherer ◽

Marcin Wozniak ◽

Pengchao Zhang ◽

...

Keyword(s):

Particle Swarm Optimization ◽

Short Term Memory ◽

Particle Swarm ◽

Electricity Consumption ◽

Attention Mechanism ◽

Short Term ◽

Swarm Optimization ◽

Term Memory ◽

Abnormal Detection ◽

Long Short Term Memory

Download Full-text

A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments

Sensors ◽

10.3390/s21041181 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1181

Author(s):

Chenhao Zhu ◽

Sheng Cai ◽

Yifan Yang ◽

Wei Xu ◽

Honghai Shen ◽

...

Keyword(s):

Kalman Filter ◽

Standard Deviation ◽

Error Compensation ◽

Random Vibration ◽

Short Term Memory ◽

Combined Method ◽

Short Term ◽

Mems Gyroscope ◽

Long Short Term Memory ◽

Lstm Network

In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.

Download Full-text

Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

Proceedings of the Institution of Mechanical Engineers Part C Journal of Mechanical Engineering Science ◽

10.1177/09544062211016505 ◽

2021 ◽

pp. 095440622110165

Author(s):

Zhang Chao ◽

Wang Wei-zhi ◽

Zhang Chen ◽

Fan Bin ◽

Wang Jian-guo ◽

...

Keyword(s):

Neural Network ◽

Fault Diagnosis ◽

Condition Monitoring ◽

Short Term Memory ◽

Vibration Signal ◽

Short Term ◽

Global Features ◽

Term Memory ◽

Long Short Term Memory ◽

Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

Download Full-text

A Method Based on Attention Mechanism using Bidirectional Long-Short Term Memory(BLSTM) for Question Answering

10.1109/icee52715.2021.9544258 ◽

2021 ◽

Author(s):

Seyed Vahid Moravvej ◽

Mohammad Javad Maleki Kahaki ◽

Moein Salimi Sartakhti ◽

Abdolreza Mirzaei

Keyword(s):

Question Answering ◽

Short Term Memory ◽

Attention Mechanism ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Download Full-text

Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks

Energies ◽

10.3390/en14185762 ◽

2021 ◽

Vol 14 (18) ◽

pp. 5762

Author(s):

Syed Basit Ali Bukhari ◽

Khawaja Khalid Mehmood ◽

Abdul Wadood ◽

Herie Park

Keyword(s):

Short Term Memory ◽

Computational Time ◽

Islanding Detection ◽

Phase Voltage ◽

Short Term ◽

Term Memory ◽

Three Phase ◽

Empirical Wavelet Transform ◽

Long Short Term Memory ◽

Lstm Network

This paper presents a new intelligent islanding detection scheme (IIDS) based on empirical wavelet transform (EWT) and long short-term memory (LSTM) network to identify islanding events in microgrids. The concept of EWT is extended to extract features from three-phase signals. First, the three-phase voltage signals sampled at the terminal of targeted distributed energy resource (DER) or point of common coupling (PCC) are decomposed into empirical modes/frequency subbands using EWT. Then, instantaneous amplitudes and instantaneous frequencies of the three-phases at different frequency subbands are combined, and various statistical features are calculated. Finally, the EWT-based features along with the three-phase voltage signals are input to the LSTM network to differentiate between non-islanding and islanding events. To assess the efficacy of the proposed IIDS, extensive simulations are performed on an IEC microgrid and an IEEE 34-node system. The simulation results verify the effectiveness of the proposed IIDS in terms of non-detection zone (NDZ), computational time, detection accuracy, and robustness against noisy measurement. Furthermore, comparisons with existing intelligent methods and different LSTM architectures demonstrate that the proposed IIDS offers higher reliability by significantly reducing the NDZ and stands robust against measurements uncertainty.

Download Full-text

Predicting Outcomes of Business Process Executions Based on LSTM Neural Networks and Attention Mechanism

10.21203/rs.3.rs-260970/v1 ◽

2021 ◽

Author(s):

Jiaojiao Wang ◽

Dongjin Yu ◽

Chengfei Liu ◽

Xiaoxiao Sun

Keyword(s):

Short Term Memory ◽

Attention Mechanism ◽

Sequential Data ◽

Time Prediction ◽

Prediction Time ◽

Highly Sensitive ◽

Long Short Term Memory ◽

Early Decision ◽

Lstm Network ◽

Hidden Layer

Abstract To effectively predict the outcome of an on-going process instance helps make an early decision, which plays an important role in so-called predictive process monitoring. Existing methods in this field are tailor-made for some empirical operations such as the prefix extraction, clustering, and encoding, leading that their relative accuracy is highly sensitive to the dataset. Moreover, they have limitations in real-time prediction applications due to the lengthy prediction time. Since Long Short-term Memory (LSTM) neural network provides a high precision in the prediction of sequential data in several areas, this paper investigates LSTM and its enhancements and proposes three different approaches to build more effective and efficient models for outcome prediction. The first move on enhancement is that we combine the original LSTM network from two directions, forward and backward, to capture more features from the completed cases. The second move on enhancement is that we add attention mechanism after extracting features in the hidden layer of LSTM network to distinct them from their attention weight. A series of extensive experiments are evaluated on twelve real datasets when comparing with other approaches. The results show that our approaches outperform the state-of-the-art ones in terms of prediction effectiveness and time performance.

Download Full-text