Early Action Prediction Using 3DCNN With LSTM and Bidirectional LSTM

2021 ◽

Vol 12 (6) ◽

pp. 2275-2281

Author(s):

Mrs. Manju D, Dr. Seetha M, Dr. Sammulal P

Keyword(s):

Neural Network ◽

Autonomous Robots ◽

Video Stream ◽

Motion Information ◽

Action Prediction ◽

Automated Recognition ◽

3 Dimensional ◽

Early Action ◽

Bidirectional Lstm ◽

Team Memory

Predicting and identifying suspicious activities before hand is highly beneficial because it results in increased protection in video surveillance cameras’. Detecting and predicting human's action before it is carried out has a variety of uses like autonomous robots, surveillance, and health care. The main focus of the paper is on automated recognition of human actions in surveillance videos. 3DCNN (3 Dimensional Convolutional Neural Network) is based on 3D convolutions, there by capturing the motion information encoded in multiple adjacent frames. The 3DCNN is combined with Long short team memory (LSTM) and Bidirectional LSTM for prediction of abnormal events from past observations of events in video stream. It is observed that 3DCNN with LSTM resulted in increased accuracy compared to 3DCNN with Bidirectional LSTM. The experiments were carried out on UCF crime Dataset.

Download Full-text

EARLY ACTION PREDICTION USING VGG16 MODEL AND BIDIRECTIONAL LSTM

INFORMATION TECHNOLOGY IN INDUSTRY ◽

10.17762/itii.v9i1.185 ◽

2021 ◽

Vol 9 (1) ◽

pp. 666-672

Author(s):

Manju D, Dr. Seetha M, Dr. Sammulal P

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Video Surveillance ◽

Early Stage ◽

Early Prediction ◽

Action Prediction ◽

Automated Driving ◽

Time Step ◽

Driving System ◽

Early Action

Action prediction plays a key function, where an expected action needs to be identified before the action is completely performed. Prediction means inferring a potential action until it occurs at its early stage. This paper emphasizes on early action prediction, to predict an action before it occurs. In real time scenarios, the early prediction can be very crucial and has many applications like automated driving system, healthcare, video surveillance and other scenarios where a proactive action is needed before the situation goes out of control. VGG16 model is used for the early action prediction which is a convolutional neural network with 16 layers depth. Besides its capability of classifying objects in the frames, the availability of model weights enhances its capability. The model weights are available freely and preferred to used in different applications or models. The VGG-16 model along with Bidirectional structure of Lstm enables the network to provide both backward and forward information at every time step. The results of the proposed approach increased observation ratio ranging from 0.1 to 1.0 compared with the accuracy of GAN model.

Download Full-text

Early Action Prediction With Generative Adversarial Networks

IEEE Access ◽

10.1109/access.2019.2904857 ◽

2019 ◽

Vol 7 ◽

pp. 35795-35804 ◽

Cited By ~ 13

Author(s):

Dong Wang ◽

Yuan Yuan ◽

Qi Wang

Keyword(s):

Generative Adversarial Networks ◽

Action Prediction ◽

Adversarial Networks ◽

Early Action

Download Full-text

DBDNet: Learning Bi-directional Dynamics for Early Action Prediction

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/126 ◽

2019 ◽

Cited By ~ 1

Author(s):

Guoliang Pang ◽

Xionghui Wang ◽

Jian-Fang Hu ◽

Qing Zhang ◽

Wei-Shi Zheng

Keyword(s):

Short Term Memory ◽

State Of The Art ◽

Action Prediction ◽

Unified Framework ◽

Short Term ◽

Term Memory ◽

Early Action ◽

Decoder Architecture ◽

Long Short Term Memory ◽

Future Direction

Predicting future actions from observed partial videos is very challenging as the missing future is uncertain and sometimes has multiple possibilities. To obtain a reliable future estimation, a novel encoder-decoder architecture is proposed for integrating the tasks of synthesizing future motions from observed videos and reconstructing observed motions from synthesized future motions in an unified framework, which can capture the bi-directional dynamics depicted in partial videos along the temporal (past-to-future) direction and reverse chronological (future-back-to-past) direction. We then employ a bi-directional long short-term memory (Bi-LSTM) architecture to exploit the learned bi-directional dynamics for predicting early actions. Our experiments on two benchmark action datasets show that learning bi-directional dynamics benefits the early action prediction and our system clearly outperforms the state-of-the-art methods.

Download Full-text