scholarly journals Action Recognition Algorithm of Spatio–Temporal Differential LSTM Based on Feature Enhancement

2021 ◽  
Vol 11 (17) ◽  
pp. 7876
Author(s):  
Kai Hu ◽  
Fei Zheng ◽  
Liguo Weng ◽  
Yiwu Ding ◽  
Junlan Jin
Keyword(s):  
Spatial Memory ◽  
Action Recognition ◽  
Short Term Memory ◽  
Human Action ◽  
Short Term ◽  
Memory State ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Spatio Temporal ◽  
Lstm Network

The Long Short-Term Memory (LSTM) network is a classic action recognition method because of its ability to extract time information. Researchers proposed many hybrid algorithms based on LSTM for human action recognition. In this paper, an improved Spatio–Temporal Differential Long Short-Term Memory (ST-D LSTM) network is proposed, an enhanced input differential feature module and a spatial memory state differential module are added to the network. Furthermore, a transmission mode of ST-D LSTM is proposed; this mode enables ST-D LSTM units to transmit the spatial memory state horizontally. Finally, these improvements are added into classical Long-term Recurrent Convolutional Networks (LRCN) to test the new network’s performance. Experimental results show that ST-D LSTM can effectively improve the accuracy of LRCN.

scholarly journals Convolutional Long Short-Term Memory Hybrid Networks for Skeletal Based Human Action Recognition

2020 ◽  
Vol 9 (3) ◽  
pp. 955-961
Keyword(s):  
Time Series ◽  
Action Recognition ◽  
Short Term Memory ◽  
Human Action Recognition ◽  
Human Action ◽  
Short Term ◽  
Term Memory ◽  
Temporal Modeling ◽  
Long Short Term Memory ◽  
Spatio Temporal

The objective is to develop a time series image representation of the skeletal action data and use it for recognition through a convolutional long short-term deep learning framework. Consequently, Kinect captured human skeletal data is transformed into a Joint Change Distance Image (JCDI) descriptor which maps the time changes in the joints. Subsequently, JCDIs are decoded spatially well with a Convolutional (CNN). Temporal decomposition is executed on long short term memory (LSTM) with data changes along x , y and z position vectors of the skeleton. We propose a combination of CNN and LSTM which maps the spatio temporal information to generate a generalized time series features for recognition. Finally, scores are fused from spatially vibrant CNNs and temporally sound LSTMs for action recognition. Publicly available action datasets such as NTU RGBD, MSR Action, UTKinect and G3D were used as test inputs for experimentation. The results showed a better performance due to spatio temporal modeling at both the representation and the recognition stages when compared to other state-of-the-arts

scholarly journals Human Action Recognition in Videos using Convolution Long Short-Term Memory Network with Spatio-Temporal Networks

2021 ◽  
Vol 5 (1) ◽  
pp. 25-33
Author(s):  
Ashok Sarabu ◽  
Ajit Kumar Santra
Keyword(s):  
Neural Networks ◽  
Action Recognition ◽  
Data Augmentation ◽  
Short Term Memory ◽  
Human Action Recognition ◽  
Human Action ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Spatio Temporal

Two-stream convolutional networks plays an essential role as a powerful feature extractor in human action recognition in videos. Recent studies have shown the importance of two-stream Convolutional Neural Networks (CNN) to recognize human action recognition. Recurrent Neural Networks (RNN) has achieved the best performance in video activity recognition combining CNN. Encouraged by CNN's results with RNN, we present a two-stream network with two CNNs and Convolution Long-Short Term Memory (CLSTM). First, we extricate Spatio-temporal features using two CNNs using pre-trained ImageNet models. Second, the results of two CNNs from step one are combined and fed as input to the CLSTM to get the overall classification score. We also explored the various fusion function performance that combines two CNNs and the effects of feature mapping at different layers. And, conclude the best fusion function along with layer number. To avoid the problem of overfitting, we adopt the data augmentation techniques. Our proposed model demonstrates a substantial improvement compared to the current two-stream methods on the benchmark datasets with 70.4% on HMDB-51 and 95.4% on UCF-101 using the pre-trained ImageNet model. Doi: 10.28991/esj-2021-01254 Full Text: PDF

scholarly journals Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 411
Author(s):  
Yunkai Zhang ◽  
Yinghong Tian ◽  
Pingyi Wu ◽  
Dongfan Chen
Keyword(s):  
Autism Spectrum Disorder ◽  
Action Recognition ◽  
Short Term Memory ◽  
Autism Spectrum ◽  
Spectrum Disorder ◽  
Short Term ◽  
Term Memory ◽  
Clinical Observations ◽  
The Neural Network ◽  
Long Short Term Memory

The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children.

scholarly journals Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

2021 ◽  
pp. 095440622110165
Author(s):  
Zhang Chao ◽  
Wang Wei-zhi ◽  
Zhang Chen ◽  
Fan Bin ◽  
Wang Jian-guo ◽  
...  
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Condition Monitoring ◽  
Short Term Memory ◽  
Vibration Signal ◽  
Short Term ◽  
Global Features ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

scholarly journals Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks

Energies ◽  
2021 ◽  
Vol 14 (18) ◽  
pp. 5762
Author(s):  
Syed Basit Ali Bukhari ◽  
Khawaja Khalid Mehmood ◽  
Abdul Wadood ◽  
Herie Park
Keyword(s):  
Short Term Memory ◽  
Computational Time ◽  
Islanding Detection ◽  
Phase Voltage ◽  
Short Term ◽  
Term Memory ◽  
Three Phase ◽  
Empirical Wavelet Transform ◽  
Long Short Term Memory ◽  
Lstm Network

This paper presents a new intelligent islanding detection scheme (IIDS) based on empirical wavelet transform (EWT) and long short-term memory (LSTM) network to identify islanding events in microgrids. The concept of EWT is extended to extract features from three-phase signals. First, the three-phase voltage signals sampled at the terminal of targeted distributed energy resource (DER) or point of common coupling (PCC) are decomposed into empirical modes/frequency subbands using EWT. Then, instantaneous amplitudes and instantaneous frequencies of the three-phases at different frequency subbands are combined, and various statistical features are calculated. Finally, the EWT-based features along with the three-phase voltage signals are input to the LSTM network to differentiate between non-islanding and islanding events. To assess the efficacy of the proposed IIDS, extensive simulations are performed on an IEC microgrid and an IEEE 34-node system. The simulation results verify the effectiveness of the proposed IIDS in terms of non-detection zone (NDZ), computational time, detection accuracy, and robustness against noisy measurement. Furthermore, comparisons with existing intelligent methods and different LSTM architectures demonstrate that the proposed IIDS offers higher reliability by significantly reducing the NDZ and stands robust against measurements uncertainty.

Sign in / Sign up

Export Citation Format

Share Document