scholarly journals Deepfake video detection: YOLO-Face convolution recurrent approach

2021 ◽  
Vol 7 ◽  
pp. e730
Author(s):  
Aya Ismail ◽  
Marwa Elpeltagy ◽  
Mervat Zaki ◽  
Kamal A. ElDahshan
Keyword(s):  
Large Scale ◽  
Operating Characteristic ◽  
Short Term Memory ◽  
Negative Impact ◽  
Characteristic Curve ◽  
Spatial Features ◽  
Temporal Features ◽  
Large Scale Dataset ◽  
Long Short Term Memory ◽  
Face Detector

Recently, the deepfake techniques for swapping faces have been spreading, allowing easy creation of hyper-realistic fake videos. Detecting the authenticity of a video has become increasingly critical because of the potential negative impact on the world. Here, a new project is introduced; You Only Look Once Convolution Recurrent Neural Networks (YOLO-CRNNs), to detect deepfake videos. The YOLO-Face detector detects face regions from each frame in the video, whereas a fine-tuned EfficientNet-B5 is used to extract the spatial features of these faces. These features are fed as a batch of input sequences into a Bidirectional Long Short-Term Memory (Bi-LSTM), to extract the temporal features. The new scheme is then evaluated on a new large-scale dataset; CelebDF-FaceForencics++ (c23), based on a combination of two popular datasets; FaceForencies++ (c23) and Celeb-DF. It achieves an Area Under the Receiver Operating Characteristic Curve (AUROC) 89.35% score, 89.38% accuracy, 83.15% recall, 85.55% precision, and 84.33% F1-measure for pasting data approach. The experimental analysis approves the superiority of the proposed method compared to the state-of-the-art methods.

scholarly journals Implementation of Deep Learning Predictor (LSTM) Algorithm for Human Mobility Prediction

2020 ◽  
Vol 14 (18) ◽  
pp. 132
Author(s):  
Ida Nurhaida ◽  
Handrie Noprisson ◽  
Vina Ayumi ◽  
Hong Wei ◽  
Erwin Dwika Putra ◽  
...  
Keyword(s):  
Large Scale ◽  
Short Term Memory ◽  
Human Mobility ◽  
Mobility Prediction ◽  
Sequential Data ◽  
Short Term ◽  
Large Scale Dataset ◽  
History Of ◽  
Long Short Term Memory ◽  
Trajectory Identification

The studies of human mobility prediction in mobile computing area gained due to the availability of large-scale dataset contained history of location trajectory. Previous work has been proposed many solutions for increasing of human mobility prediction result accuration, however, only few researchers have addressed the issue of<em> </em>human mobility for implementation of LSTM networks. This study attempted to use classical methodologies by combining LSTM and DBSCAN because those algorithms can tackle problem in human mobility, including large-scale sequential data modeling and number of clusters of arbitrary trajectory identification. The method of research consists of DBSCAN for clustering, long short-term memory (LSTM) algorithm for modelling and prediction, and Root Mean Square Error (RMSE) for evaluation. As the result,<em> </em>the prediction error or RMSE value reached score 3.551 by setting LSTM with parameter of <em>epoch</em> and <em>batch_size</em> is 100 and 20 respectively.

scholarly journals Abnormal Activity Recognition from Surveillance Videos Using Convolutional Neural Network

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8291
Author(s):  
Shabana Habib ◽  
Altaf Hussain ◽  
Waleed Albattah ◽  
Muhammad Islam ◽  
Sheroz Khan ◽  
...  
Keyword(s):  
Short Term Memory ◽  
Law Enforcement Agencies ◽  
Surveillance Videos ◽  
Spatial Features ◽  
Temporal Features ◽  
Proposed Model ◽  
Enormous Amount ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Activity Method

Background and motivation: Every year, millions of Muslims worldwide come to Mecca to perform the Hajj. In order to maintain the security of the pilgrims, the Saudi government has installed about 5000 closed circuit television (CCTV) cameras to monitor crowd activity efficiently. Problem: As a result, these cameras generate an enormous amount of visual data through manual or offline monitoring, requiring numerous human resources for efficient tracking. Therefore, there is an urgent need to develop an intelligent and automatic system in order to efficiently monitor crowds and identify abnormal activity. Method: The existing method is incapable of extracting discriminative features from surveillance videos as pre-trained weights of different architectures were used. This paper develops a lightweight approach for accurately identifying violent activity in surveillance environments. As the first step of the proposed framework, a lightweight CNN model is trained on our own pilgrim’s dataset to detect pilgrims from the surveillance cameras. These preprocessed salient frames are passed to a lightweight CNN model for spatial features extraction in the second step. In the third step, a Long Short Term Memory network (LSTM) is developed to extract temporal features. Finally, in the last step, in the case of violent activity or accidents, the proposed system will generate an alarm in real time to inform law enforcement agencies to take appropriate action, thus helping to avoid accidents and stampedes. Results: We have conducted multiple experiments on two publicly available violent activity datasets, such as Surveillance Fight and Hockey Fight datasets; our proposed model achieved accuracies of 81.05 and 98.00, respectively.

scholarly journals A combined convolutional and recurrent neural network for enhanced glaucoma detection

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Soheila Gheisari ◽  
Sahar Shariflou ◽  
Jack Phu ◽  
Paul J. Kennedy ◽  
Ashish Agar ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Fundus Images ◽  
Spatial Features ◽  
Glaucoma Detection ◽  
Temporal Features ◽  
Long Short Term Memory ◽  
Sequential Images ◽  
F Measure

AbstractGlaucoma, a leading cause of blindness, is a multifaceted disease with several patho-physiological features manifesting in single fundus images (e.g., optic nerve cupping) as well as fundus videos (e.g., vascular pulsatility index). Current convolutional neural networks (CNNs) developed to detect glaucoma are all based on spatial features embedded in an image. We developed a combined CNN and recurrent neural network (RNN) that not only extracts the spatial features in a fundus image but also the temporal features embedded in a fundus video (i.e., sequential images). A total of 1810 fundus images and 295 fundus videos were used to train a CNN and a combined CNN and Long Short-Term Memory RNN. The combined CNN/RNN model reached an average F-measure of 96.2% in separating glaucoma from healthy eyes. In contrast, the base CNN model reached an average F-measure of only 79.2%. This proof-of-concept study demonstrates that extracting spatial and temporal features from fundus videos using a combined CNN and RNN, can markedly enhance the accuracy of glaucoma detection.

Composite load modeling by spatial-temporal deep attention network based on wide-area monitoring systems

2021 ◽  
pp. 1-12
Author(s):  
Omid Izadi Ghafarokhi ◽  
Mazda Moattari ◽  
Ahmad Forouzantabar
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Wide Area ◽  
Load Modeling ◽  
Wide Area Monitoring ◽  
Training Performance ◽  
Spatial Features ◽  
Temporal Features ◽  
Frequency Domains ◽  
Area Monitoring

With the development of the wide-area monitoring system (WAMS), power system operators are capable of providing an accurate and fast estimation of time-varying load parameters. This study proposes a spatial-temporal deep network-based new attention concept to capture the dynamic and static patterns of electrical load consumption through modeling complicated and non-stationary interdependencies between time sequences. The designed deep attention-based network benefits from long short-term memory (LSTM) based component to learning temporal features in time and frequency-domains as encoder-decoder based recurrent neural network. Furthermore, to inherently learn spatial features, a convolutional neural network (CNN) based attention mechanism is developed. Besides, this paper develops a loss function based on a pseudo-Huber concept to enhance the robustness of the proposed network in noisy conditions as well as improve the training performance. The simulation results on IEEE 68-bus demonstrates the effectiveness and superiority of the proposed network through comparison with several previously presented and state-of-the-art methods.

scholarly journals Social Risk Early Warning of Environmental Damage of Large-Scale Construction Projects in China Based on Network Governance and LSTM Model

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-13 ◽  
Author(s):  
Junmin Fang ◽  
Dechun Huang ◽  
Jingrong Xu
Keyword(s):  
Early Warning ◽  
Construction Projects ◽  
Large Scale ◽  
Short Term Memory ◽  
Network Governance ◽  
Scale Construction ◽  
Environmental Damage ◽  
Social Risk ◽  
Short Term ◽  
Long Short Term Memory

With the improvement of citizens’ risk perception ability and environmental protection awareness, social conflicts caused by environmental problems in large-scale construction projects are becoming more and more frequent. Traditional social risk prevention management has some defects in obtaining risk data, such as limited coverage, poor availability, and insufficient timeliness, which makes it impossible to realize effective early warning of social risks in the era of big data. This paper focuses on the three environments of diversification of stakeholders, risk media, and big data era. The evolution characteristics of the social risk of environmental damage of large-scale construction projects are analyzed from the four stages of incubation, outbreak, mitigation, and regression in essence. On this basis, a social risk early warning model is constructed， and the multicenter network governance mode of social risk of environmental damage in large-scale construction projects and practical social risk prevention strategies in different stages are put forward. Experiments show that the long short-term memory neural network model is effective and feasible for predicting the social risk trend of environmental damage of large-scale construction projects. Compared with other classical models, the long short-term memory model has the advantages of strong processing capability and high early warning accuracy for time-sensitive data and will have broad application prospects in the field of risk control research. By using the network governance framework and long short-term memory model, this paper studies the environmental mass events of large-scale construction projects on the risk early warning method, providing reference for the government to effectively prevent and control social risk of environmental damage of large-scale construction project in China.

scholarly journals Validating the validation: reanalyzing a large-scale comparison of deep learning and machine learning models for bioactivity prediction

2020 ◽  
Vol 34 (7) ◽  
pp. 717-730 ◽  
Author(s):  
Matthew C. Robinson ◽  
Robert C. Glen ◽  
Alpha A. Lee
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Numerical Experiments ◽  
Large Scale ◽  
Operating Characteristic ◽  
Characteristic Curve ◽  
Learning Models ◽  
Bioactivity Prediction ◽  
Operating Characteristic Curve ◽  
Machine Learning Models

Abstract Machine learning methods may have the potential to significantly accelerate drug discovery. However, the increasing rate of new methodological approaches being published in the literature raises the fundamental question of how models should be benchmarked and validated. We reanalyze the data generated by a recently published large-scale comparison of machine learning models for bioactivity prediction and arrive at a somewhat different conclusion. We show that the performance of support vector machines is competitive with that of deep learning methods. Additionally, using a series of numerical experiments, we question the relevance of area under the receiver operating characteristic curve as a metric in virtual screening. We further suggest that area under the precision–recall curve should be used in conjunction with the receiver operating characteristic curve. Our numerical experiments also highlight challenges in estimating the uncertainty in model performance via scaffold-split nested cross validation.

scholarly journals Dynamic prediction of cardiovascular disease using improved LSTM

2019 ◽  
Vol 3 (1) ◽  
pp. 14-25
Author(s):  
Kuang Junwei ◽  
Hangzhou Yang ◽  
Liu Junjiang ◽  
Yan Zhijun
Keyword(s):  
Cardiovascular Disease ◽  
Large Scale ◽  
Prediction Models ◽  
Short Term Memory ◽  
Classification Performance ◽  
Time Interval ◽  
Dynamic Prediction ◽  
Content Type ◽  
Period Data ◽  
Long Short Term Memory

Purpose Previous dynamic prediction models rarely handle multi-period data with different intervals, and the large-scale patient hospital records are not effectively used to improve the prediction performance. This paper aims to focus on the prediction of cardiovascular disease using the improved long short-term memory (LSTM) model. Design/methodology/approach A new model based on the traditional LSTM was proposed to predict cardiovascular disease. The irregular time interval is smoothed to obtain the time parameter vector, and it is used as the input of the forgetting gate of LSTM to overcome the prediction obstacle caused by the irregular time interval. Findings The experimental results show that the dynamic prediction model proposed in this paper obtained a significant better classification performance compared with the traditional LSTM model. Originality/value In this paper, the authors improved the LSTM by smoothing the irregular time between different medical stages of the patient to obtain the temporal feature vector.

scholarly journals Multi-Scale Remote Sensing Semantic Analysis Based on a Global Perspective

2019 ◽  
Vol 8 (9) ◽  
pp. 417 ◽  
Author(s):  
Wei Cui ◽  
Dongyou Zhang ◽  
Xin He ◽  
Meng Yao ◽  
Ziwei Wang ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Receptive Field ◽  
Large Scale ◽  
Semantic Information ◽  
Short Term Memory ◽  
Global Perspective ◽  
Small Scale ◽  
Multi Scale ◽  
Memory Network ◽  
Long Short Term Memory

Remote sensing image captioning involves remote sensing objects and their spatial relationships. However, it is still difficult to determine the spatial extent of a remote sensing object and the size of a sample patch. If the patch size is too large, it will include too many remote sensing objects and their complex spatial relationships. This will increase the computational burden of the image captioning network and reduce its precision. If the patch size is too small, it often fails to provide enough environmental and contextual information, which makes the remote sensing object difficult to describe. To address this problem, we propose a multi-scale semantic long short-term memory network (MS-LSTM). The remote sensing images are paired into image patches with different spatial scales. First, the large-scale patches have larger sizes. We use a Visual Geometry Group (VGG) network to extract the features from the large-scale patches and input them into the improved MS-LSTM network as the semantic information, which provides a larger receptive field and more contextual semantic information for small-scale image caption so as to play the role of global perspective, thereby enabling the accurate identification of small-scale samples with the same features. Second, a small-scale patch is used to highlight remote sensing objects and simplify their spatial relations. In addition, the multi-receptive field provides perspectives from local to global. The experimental results demonstrated that compared with the original long short-term memory network (LSTM), the MS-LSTM’s Bilingual Evaluation Understudy (BLEU) has been increased by 5.6% to 0.859, thereby reflecting that the MS-LSTM has a more comprehensive receptive field, which provides more abundant semantic information and enhances the remote sensing image captions.

The performance of LSTM models from basin to continental scales

2020 ◽  
Author(s):  
Frederik Kratzert ◽  
Daniel Klotz ◽  
Günter Klambauer ◽  
Grey Nearing ◽  
Sepp Hochreiter
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Short Term Memory ◽  
Regional Scale ◽  
Data Driven ◽  
Hydrological Models ◽  
Short Term ◽  
Term Memory ◽  
Basin Scale ◽  
Long Short Term Memory

&lt;p&gt;Simulation accuracy among traditional hydrological models usually degrades significantly when going from single basin to regional scale. Hydrological models perform best when calibrated for specific basins, and do worse when a regional calibration scheme is used.&amp;#160;&lt;/p&gt;&lt;p&gt;One reason for this is that these models do not (have to) learn hydrological processes from data. Rather, they have a predefined model structure and only a handful of parameters adapt to specific basins. This often yields less-than-optimal parameter values when the loss is not determined by a single basin, but by many through regional calibration.&lt;/p&gt;&lt;p&gt;The opposite is true for data driven approaches where models tend to get better with more and diverse training data. We examine whether this holds true when modeling rainfall-runoff processes with deep learning, or if, like their process-based counterparts, data-driven hydrological models degrade when going from basin to regional scale.&lt;/p&gt;&lt;p&gt;Recently, Kratzert et al. (2018) showed that the Long Short-Term Memory network (LSTM), a special type of recurrent neural network, achieves comparable performance to the SAC-SMA at basin scale. In follow up work Kratzert et al. (2019a) trained a single LSTM for hundreds of basins in the continental US, which outperformed a set of hydrological models significantly, even compared to basin-calibrated hydrological models. On average, a single LSTM is even better in out-of-sample predictions (ungauged) compared to the SAC-SMA in-sample (gauged) or US National Water Model (Kratzert et al. 2019b).&lt;/p&gt;&lt;p&gt;LSTM-based approaches usually involve tuning a large number of hyperparameters, such as the number of neurons, number of layers, and learning rate, that are critical for the predictive performance. Therefore, large-scale hyperparameter search has to be performed to obtain a proficient LSTM network.&amp;#160;&amp;#160;&lt;/p&gt;&lt;p&gt;However, in the abovementioned studies, hyperparameter optimization was not conducted at large scale and e.g. in Kratzert et al. (2018) the same network hyperparameters were used in all basins, instead of tuning hyperparameters for each basin separately. It is yet unclear whether LSTMs follow the same trend of traditional hydrological models to degrade performance from basin to regional scale.&amp;#160;&lt;/p&gt;&lt;p&gt;In the current study, we performed a computational expensive, basin-specific hyperparameter search to explore how site-specific LSTMs differ in performance compared to regionally calibrated LSTMs. We compared our results to the mHM and VIC models, once calibrated per-basin and once using an MPR regionalization scheme. These benchmark models were calibrated individual research groups, to eliminate bias in our study. We analyse whether differences in basin-specific vs regional model performance can be linked to basin attributes or data set characteristics.&lt;/p&gt;&lt;p&gt;References:&lt;/p&gt;&lt;p&gt;Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall&amp;#8211;runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005&amp;#8211;6022, https://doi.org/10.5194/hess-22-6005-2018, 2018.&amp;#160;&lt;/p&gt;&lt;p&gt;Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., and Nearing, G.: Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets, Hydrol. Earth Syst. Sci., 23, 5089&amp;#8211;5110, https://doi.org/10.5194/hess-23-5089-2019, 2019a.&amp;#160;&lt;/p&gt;&lt;p&gt;Kratzert, F., Klotz, D., Herrnegger, M., Sampson, A. K., Hochreiter, S., &amp; Nearing, G. S.: Toward improved predictions in ungauged basins: Exploiting the power of machine learning. Water Resources Research, 55. https://doi.org/10.1029/2019WR026065, 2019b.&lt;/p&gt;

Sign in / Sign up

Export Citation Format

Share Document