scholarly journals Data-Driven Stability Assessment of Multilayer Long Short-Term Memory Networks

2021 ◽  
Vol 11 (4) ◽  
pp. 1829
Author(s):  
Davide Grande ◽  
Catherine A. Harris ◽  
Giles Thomas ◽  
Enrico Anderlini
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Short Term Memory ◽  
Moving Average ◽  
Machine Learning Algorithms ◽  
Physical Models ◽  
Data Driven ◽  
The Stability ◽  
And Control ◽  
Nonlinear Autoregressive

Recurrent Neural Networks (RNNs) are increasingly being used for model identification, forecasting and control. When identifying physical models with unknown mathematical knowledge of the system, Nonlinear AutoRegressive models with eXogenous inputs (NARX) or Nonlinear AutoRegressive Moving-Average models with eXogenous inputs (NARMAX) methods are typically used. In the context of data-driven control, machine learning algorithms are proven to have comparable performances to advanced control techniques, but lack the properties of the traditional stability theory. This paper illustrates a method to prove a posteriori the stability of a generic neural network, showing its application to the state-of-the-art RNN architecture. The presented method relies on identifying the poles associated with the network designed starting from the input/output data. Providing a framework to guarantee the stability of any neural network architecture combined with the generalisability properties and applicability to different fields can significantly broaden their use in dynamic systems modelling and control.

Study on the Smart Diagnosis of Gas Turbine With Distributed Computing Technique

2019 ◽  
Author(s):  
Hang Wu ◽  
Jinwei Chen ◽  
Huisheng Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Gas Turbine ◽  
Short Term Memory ◽  
Computing Time ◽  
Physical Models ◽  
Data Driven ◽  
Computing Technique ◽  
Training Time ◽  
High Level

Abstract Monitoring and diagnosis of a gas turbine is a critical issue in equipment maintenance field. Traditional diagnosis methods are established on the basis of physical models. However, the complexity and degradation of gas turbine limit both comprehensiveness and accuracy of these physical models, making the diagnosis less effective. Therefore, data-driven models are introduced to supplement and revise previous models. Benefitting from the prosperous development of machine learning, neural network has been greatly improved and widely used in various fields of data mining. Three neural networks, Multilayer Perceptron, Convolutional Neural Network and Long Short-term Memory Network are applied in data-driven model establishment. Their training time and prediction accuracy are the two most important factors in judging the effectiveness. An active real time training which means training and predicting simultaneously is applied as the main modelling method for an on-line diagnosis system. Three periods are defined according to the time line: data preparation period, model establishing period and stable prediction period. From the three above neural networks, the most effective data-driven models that corresponding to the last two periods are tested and selected, the purpose is to ensure the high level of accuracy. When high level of accuracy is demanded, neural network always need large computing time and memory space in data learning process. To avoid prediction delay and keep rapid response for the coming fault, distributed training on a 1-master 2-workers computer cluster is designed and applied in this system. Two types of data parallelism are realized on the cluster through Apache Spark and Shell Script for Linux. Comparing with each other and the local training mode, the results shows that dispensing data at first and averaging parameters at last reaches a better outcome both in high accuracy and low training time.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Real-Time Detection of Dictionary DGA Network Traffic Using Deep Learning

2021 ◽  
Vol 2 (2) ◽  
Author(s):  
Kate Highnam ◽  
Domenic Puzio ◽  
Song Luo ◽  
Nicholas R. Jennings
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real Time ◽  
Network Traffic ◽  
Short Term Memory ◽  
Domain Names ◽  
Control Networks ◽  
Detection Techniques ◽  
Lstm Network ◽  
And Control

AbstractBotnets and malware continue to avoid detection by static rule engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the “bagging” model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a convolutional neural network (CNN) and a long short-term memory (LSTM) network for DGA detection. Our unique architecture is found to be the most consistent in performance in terms of AUC, $$F_1$$ F 1 score, and accuracy when generalising across different dictionary DGA classification tasks compared to current state-of-the-art deep learning architectures. We validate using reverse-engineered dictionary DGA domains and detail our real-time implementation strategy for scoring real-world network logs within a large enterprise. In 4 h of actual network traffic, the model discovered at least five potential command-and-control networks that commercial vendor tools did not flag.

scholarly journals FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos

2020 ◽  
Vol 15 (5) ◽  
pp. 791-801 ◽  
Author(s):  
Sophia Bano ◽  
Francisco Vasconcelos ◽  
Emmanuel Vander Poorten ◽  
Tom Vercauteren ◽  
Sebastien Ourselin ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Laser Photocoagulation ◽  
Short Term Memory ◽  
Total Duration ◽  
Frame Rate ◽  
Superior Performance ◽  
Computer Assisted ◽  
Convolutional Network ◽  
Spatio Temporal

Abstract Purpose Fetoscopic laser photocoagulation is a minimally invasive surgery for the treatment of twin-to-twin transfusion syndrome (TTTS). By using a lens/fibre-optic scope, inserted into the amniotic cavity, the abnormal placental vascular anastomoses are identified and ablated to regulate blood flow to both fetuses. Limited field-of-view, occlusions due to fetus presence and low visibility make it difficult to identify all vascular anastomoses. Automatic computer-assisted techniques may provide better understanding of the anatomical structure during surgery for risk-free laser photocoagulation and may facilitate in improving mosaics from fetoscopic videos. Methods We propose FetNet, a combined convolutional neural network (CNN) and long short-term memory (LSTM) recurrent neural network architecture for the spatio-temporal identification of fetoscopic events. We adapt an existing CNN architecture for spatial feature extraction and integrated it with the LSTM network for end-to-end spatio-temporal inference. We introduce differential learning rates during the model training to effectively utilising the pre-trained CNN weights. This may support computer-assisted interventions (CAI) during fetoscopic laser photocoagulation. Results We perform quantitative evaluation of our method using 7 in vivo fetoscopic videos captured from different human TTTS cases. The total duration of these videos was 5551 s (138,780 frames). To test the robustness of the proposed approach, we perform 7-fold cross-validation where each video is treated as a hold-out or test set and training is performed using the remaining videos. Conclusion FetNet achieved superior performance compared to the existing CNN-based methods and provided improved inference because of the spatio-temporal information modelling. Online testing of FetNet, using a Tesla V100-DGXS-32GB GPU, achieved a frame rate of 114 fps. These results show that our method could potentially provide a real-time solution for CAI and automating occlusion and photocoagulation identification during fetoscopic procedures.

scholarly journals Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

2020 ◽  
Vol 27 (3) ◽  
pp. 373-389 ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
Pedram Hassanzadeh ◽  
Devika Subramanian
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Short Term Memory ◽  
Data Driven ◽  
Reservoir Computing ◽  
Short Term ◽  
Term Memory ◽  
Machine Learning Methods ◽  
Long Short Term Memory ◽  
Lorenz 96

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

Modelling and control of greenhouse system using neural networks

2016 ◽  
Vol 40 (3) ◽  
pp. 918-929 ◽  
Author(s):  
A Manonmani ◽  
T Thyagarajan ◽  
M Elango ◽  
S Sutha
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Moving Average ◽  
Growth Conditions ◽  
Operating Conditions ◽  
Adverse Weather ◽  
Control Schemes ◽  
Non Linear ◽  
Auto Regressive ◽  
And Control

A greenhouse system (GHS) is a closed structure that facilitates modified growth conditions to crops and provides protection from pests, diseases and adverse weather. However, a GHS exhibits non-linearity due to the interaction between the biological subsystem and the physical subsystem. Non-linear systems are difficult to control, particularly when their characteristics change with time. These systems are best handled with methods of computation intelligence, such as artificial neural networks (ANNs) and fuzzy systems. In the present work, the approximation capability of a neural network is used to model and control sufficient growth conditions of a GHS. An optimal neural network-based non-linear auto regressive with exogenous input (NARX) time series model is developed for a GHS. Based on the NARX model, two intelligent control schemes, namely a neural predictive controller (NPC) and non-linear auto regressive moving average (NARMA-L2) controller are proposed to achieve the desired growth conditions such as humidity and temperature for a better yield. Finally, closed-loop performances of the above two control schemes for servo and regulatory operations are analysed for various operating conditions using performance indices.

IDENTIFICATION OF SERVICE TRAFFIC IN IMT-2020 AND SUBSEQUENT GENERATION COMMUNICATION NETWORKS BASED ON FLOW METADATA AND MACHINE LEARNING ALGORITHMS

2020 ◽  
Author(s):  
А.Н. ВОЛКОВ ◽  
А.Е. КУЧЕРЯВЫЙ
Keyword(s):  
Neural Network ◽  
Communication Networks ◽  
Machine Learning Algorithms ◽  
Laboratory Model ◽  
Software Defined Network ◽  
Subsequent Generation ◽  
Traffic Forecasting ◽  
Network Analytics ◽  
Practical Research ◽  
And Control

Предлагается новый метод идентификации трафика на основе нейросетевой аналитики метаданных потоков для последующей аналитики прогнозирования и управления трафиком с учетом возможностей программируемости сетей SDN/ NFV. Дано обоснование выбора метода идентификации, основанного на алгоритмах искусственного интеллекта, и показаны его преимущества перед другими методами. Для апробации предложенного метода разработано программное обеспечение и проведены практические исследования на сегменте лабораторной модельной программно-конфигурируемой сети. This article proposes a new method of identifying traffic based on neural network analytics of flow metadata for subsequent analytics of traffic forecasting and control taking into account the programmability of SDN/NFV networks. The paper provides a rationale for the choice of the identification method based on artificial intelligence algorithms and shows its advantages over other methods. To test the proposed method, the software was developed and practical research was carried out on a segment of a laboratory model software-defined network.

A recurrent neural network-based predictive model for pressure ulcers based on an intensive care unit dataset (MIMIC-III) (Preprint)

2021 ◽  
Author(s):  
Minseop Park ◽  
Hyeok Choi ◽  
Hee-Sung Ahn ◽  
Hee-Ju Kang ◽  
Saehoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Intensive Care Unit ◽  
Intensive Care ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Machine Learning Algorithms ◽  
Dynamic Features ◽  
Braden Scale ◽  
Mlp Model ◽  
Mimic Iii

BACKGROUND A pressure ulcer (PU) is a localized cutaneous injury caused by pressure or shear, which usually occurs in the region of a bony prominence. PUs are common in hospitalized patients and cause complications including infection. OBJECTIVE This study aimed to build a recurrent neural network-based algorithm to predict PUs 24 hours before their occurrence. METHODS This study analyzed a freely accessible intensive care unit (ICU) dataset, MIMIC- III. Deep learning and machine learning algorithms including long short-term memory (LSTM), multilayer perceptron (MLP), and XGBoost were applied to 37 dynamic features (including the Braden scale, vital signs and laboratory results, and interventions to reduce the risk of PUs) and 35 static features (including the length of time spent in the ICU, demographics, and comorbidities). Their outcomes were compared in terms of the area under the receiver operating characteristic (AUROC) and the area under the precision-recall curve (AUPRC). RESULTS A total of 1,048 cases of PUs (10.0%) and 9,402 controls (90.0%) without PUs satisfied the inclusion criteria for analysis. The LSTM + MLP model (AUROC: 0.7929 ± 0.0095, AUPRC: 0.4819 ± 0.0109) outperformed the other models, namely: MLP model (AUROC: 0.7777 ± 0.0083, AUPRC: 0.4527 ± 0.0195) and XGBoost (AUROC: 0.7465 ± 0.0087, AUPRC: 0.4052 ± 0.0087). Various features, including the length of time spent in the ICU, Glasgow coma scale, and the Braden scale, contributed to the prediction model. CONCLUSIONS This study suggests that recurrent neural network-based algorithms such as LSTM can be applied to evaluate the risk of PUs in ICU patients.

Sign in / Sign up

Export Citation Format

Share Document