Self-Attention Network for Partial-Discharge Diagnosis in Gas-Insulated Switchgear

Detecting, measuring, and classifying partial discharges (PDs) are important tasks for assessing the condition of insulation systems used in different electrical equipment. Owing to the implementation of the phase-resolved PD (PRPD) as a sequence input, an existing method that processes sequential data, e.g., the recurrent neural network, using a long short-term memory (LSTM) has been applied for fault classification. However, the model performance is not further improved because of the lack of supporting parallel computation and the inability to recognize the relevance of all inputs. To overcome these two drawbacks, we propose a novel deep-learning model in this study based on a self-attention mechanism to classify the PD patterns in a gas-insulated switchgear (GIS). The proposed model uses a self-attention block that offers the advantages of simultaneous computation and selective focusing on parts of the PRPD signals and a classification block to finally classify faults in the GIS. Moreover, the combination of LSTM and self-attention is considered for comparison purposes. The experimental results show that the proposed method achieves performance superiority compared with the previous neural networks, whereas the model complexity is significantly reduced.

Download Full-text

Optimising Deep Learning at the Edge for Accurate Hourly Air Quality Prediction

Sensors ◽

10.3390/s21041064 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1064

Author(s):

I Nyoman Kusuma Wardana ◽

Julian W. Gardner ◽

Suhaib A. Fahmy

Keyword(s):

Deep Learning ◽

Air Quality ◽

Short Term Memory ◽

Model Performance ◽

Learning Model ◽

Sensor Data ◽

Raspberry Pi ◽

Learning Models ◽

Proposed Model ◽

Deep Learning Model

Accurate air quality monitoring requires processing of multi-dimensional, multi-location sensor data, which has previously been considered in centralised machine learning models. These are often unsuitable for resource-constrained edge devices. In this article, we address this challenge by: (1) designing a novel hybrid deep learning model for hourly PM2.5 pollutant prediction; (2) optimising the obtained model for edge devices; and (3) examining model performance running on the edge devices in terms of both accuracy and latency. The hybrid deep learning model in this work comprises a 1D Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) to predict hourly PM2.5 concentration. The results show that our proposed model outperforms other deep learning models, evaluated by calculating RMSE and MAE errors. The proposed model was optimised for edge devices, the Raspberry Pi 3 Model B+ (RPi3B+) and Raspberry Pi 4 Model B (RPi4B). This optimised model reduced file size to a quarter of the original, with further size reduction achieved by implementing different post-training quantisation. In total, 8272 hourly samples were continuously fed to the edge device, with the RPi4B executing the model twice as fast as the RPi3B+ in all quantisation modes. Full-integer quantisation produced the lowest execution time, with latencies of 2.19 s and 4.73 s for RPi4B and RPi3B+, respectively.

Download Full-text

Long Short-Term Memory with Dynamic Skip Connections

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33016481 ◽

2019 ◽

Vol 33 ◽

pp. 6481-6488 ◽

Cited By ~ 3

Author(s):

Tao Gui ◽

Qi Zhang ◽

Lujun Zhao ◽

Yaosong Lin ◽

Minlong Peng ◽

...

Keyword(s):

Language Processing ◽

Short Term Memory ◽

Training Data ◽

Sequential Data ◽

Short Term ◽

Term Memory ◽

Transition Functions ◽

Proposed Model ◽

Long Short Term Memory

In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length. However, LSTM can still experience difficulty in capturing long-term dependencies. In this work, we tried to alleviate this problem by introducing a dynamic skip connection, which can learn to directly connect two dependent words. Since there is no dependency information in the training data, we propose a novel reinforcement learning-based method to model the dependency relationship and connect dependent words. The proposed model computes the recurrent transition functions based on the skip connections, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially. Our experimental results on three natural language processing tasks demonstrate that the proposed method can achieve better performance than existing methods. In the number prediction experiment, the proposed model outperformed LSTM with respect to accuracy by nearly 20%.

Download Full-text

A Multistep Prediction of Hydropower Station Inflow Based on Bagging-LSTM Model

Discrete Dynamics in Nature and Society ◽

10.1155/2021/1031442 ◽

2021 ◽

Vol 2021 ◽

pp. 1-12

Author(s):

Lulu Wang ◽

Hanmei Peng ◽

Mao Tan ◽

Rui Pan

Keyword(s):

Short Term Memory ◽

Southern China ◽

Time Lag ◽

Weather Data ◽

Hydropower Station ◽

Related Factors ◽

Joint Influence ◽

Proposed Model ◽

Long Short Term Memory ◽

Deep Learning Model

The inflow forecasting is one of the most important technologies for modern hydropower station. Under the joint influence of soil, upstream inflow, and precipitation, the inflow is often characterized by time lag, nonlinearity, and uncertainty and then results in the difficulty of accurate multistep prediction of inflow. To address the coupling relationship between inflow and the related factors, this paper proposes a long short-term memory deep learning model based on the Bagging algorithm (Bagging-LSTM) to predict the inflows of future 3 h, 12 h, and 24 h, respectively. To validate the proposed model, the inflow and related weather data come from a hydropower station in southern China. Compared with the classical time series models, the results show that the proposed model outperforms them on different accuracy metrics, especially in the scenario of multistep prediction.

Download Full-text

Hybrid deep learning model using recurrent neural network and gated recurrent unit for heart disease prediction

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v11i6.pp5467-5476 ◽

2021 ◽

Vol 11 (6) ◽

pp. 5467

Author(s):

Surenthiran Krishnan ◽

Pritheega Magalingam ◽

Roslina Ibrahim

Keyword(s):

Neural Network ◽

Heart Disease ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Learning Model ◽

Disease Prediction ◽

The Neural Network ◽

Proposed Model ◽

Deep Learning Model

<span>This paper proposes a new hybrid deep learning model for heart disease prediction using recurrent neural network (RNN) with the combination of multiple gated recurrent units (GRU), long short-term memory (LSTM) and Adam optimizer. This proposed model resulted in an outstanding accuracy of 98.6876% which is the highest in the existing model of RNN. The model was developed in Python 3.7 by integrating RNN in multiple GRU that operates in Keras and Tensorflow as the backend for deep learning process, supported by various Python libraries. The recent existing models using RNN have reached an accuracy of 98.23% and deep neural network (DNN) has reached 98.5%. The common drawbacks of the existing models are low accuracy due to the complex build-up of the neural network, high number of neurons with redundancy in the neural network model and imbalance datasets of Cleveland. Experiments were conducted with various customized model, where results showed that the proposed model using RNN and multiple GRU with synthetic minority oversampling technique (SMOTe) has reached the best performance level. This is the highest accuracy result for RNN using Cleveland datasets and much promising for making an early heart disease prediction for the patients.</span>

Download Full-text

A Novel Hybrid Deep Learning Model for Detecting COVID-19-Related Rumors on Social Media Based on LSTM and Concatenated Parallel CNNs

Applied Sciences ◽

10.3390/app11177940 ◽

2021 ◽

Vol 11 (17) ◽

pp. 7940

Author(s):

Mohammed Al-Sarem ◽

Abdullah Alsaeedi ◽

Faisal Saeed ◽

Wadii Boulila ◽

Omair AmeerBakhsh

Keyword(s):

Machine Learning ◽

Social Media ◽

Deep Learning ◽

Short Term Memory ◽

Learning Model ◽

Superior Performance ◽

Detection Methods ◽

The Internet ◽

Proposed Model ◽

Deep Learning Model

Spreading rumors in social media is considered under cybercrimes that affect people, societies, and governments. For instance, some criminals create rumors and send them on the internet, then other people help them to spread it. Spreading rumors can be an example of cyber abuse, where rumors or lies about the victim are posted on the internet to send threatening messages or to share the victim’s personal information. During pandemics, a large amount of rumors spreads on social media very fast, which have dramatic effects on people’s health. Detecting these rumors manually by the authorities is very difficult in these open platforms. Therefore, several researchers conducted studies on utilizing intelligent methods for detecting such rumors. The detection methods can be classified mainly into machine learning-based and deep learning-based methods. The deep learning methods have comparative advantages against machine learning ones as they do not require preprocessing and feature engineering processes and their performance showed superior enhancements in many fields. Therefore, this paper aims to propose a Novel Hybrid Deep Learning Model for Detecting COVID-19-related Rumors on Social Media (LSTM–PCNN). The proposed model is based on a Long Short-Term Memory (LSTM) and Concatenated Parallel Convolutional Neural Networks (PCNN). The experiments were conducted on an ArCOV-19 dataset that included 3157 tweets; 1480 of them were rumors (46.87%) and 1677 tweets were non-rumors (53.12%). The findings of the proposed model showed a superior performance compared to other methods in terms of accuracy, recall, precision, and F-score.

Download Full-text

One-Shot Learning for Partial Discharge Diagnosis Using Ultra-High-Frequency Sensor in Gas-Insulated Switchgear

Sensors ◽

10.3390/s20195562 ◽

2020 ◽

Vol 20 (19) ◽

pp. 5562

Author(s):

Vo-Nguyen Tuyet-Doan ◽

The-Duong Do ◽

Ngoc-Diem Tran-Thi ◽

Young-Woo Youn ◽

Yong-Hwa Kim

Keyword(s):

Deep Learning ◽

Fault Diagnosis ◽

High Frequency ◽

Electrical Equipment ◽

Training Data ◽

Ultra High Frequency ◽

Frequency Sensor ◽

Gas Insulated Switchgear ◽

Insulation Systems ◽

Metric Function

In recent years, deep learning has been successfully used in order to classify partial discharges (PDs) for assessing the condition of insulation systems in different electrical equipment. However, fault diagnosis using deep learning is still challenging, as it requires a large amount of training data, which is difficult and expensive to obtain in the real world. This paper proposes a novel one-shot learning method for fault diagnosis using a small dataset of phase-resolved PDs (PRPDs) in a gas-insulated switchgear (GIS). The proposed method is based on a Siamese network framework, which employs a distance metric function for predicting sample pairs from the same PRPD class or different PRPD classes. Experimental results over the small PRPD dataset that was obtained from an ultra-high-frequency sensor in the GIS show that the proposed method achieves outstanding performance for PRPD fault diagnosis as compared with the previous methods.

Download Full-text

A Deep Neural Network Model for the Detection and Classification of Emotions from Textual Content

Complexity ◽

10.1155/2022/8221121 ◽

2022 ◽

Vol 2022 ◽

pp. 1-12

Author(s):

Muhammad Zubair Asghar ◽

Adidah Lajis ◽

Muhammad Mansoor Alam ◽

Mohd Khairil Rahmat ◽

Haidawati Mohamad Nasir ◽

...

Keyword(s):

Product Information ◽

Short Term Memory ◽

User Behavior ◽

Automated Identification ◽

Online Social Media ◽

Proposed Model ◽

Textual Content ◽

Deep Learning Model

Emotion-based sentimental analysis has recently received a lot of interest, with an emphasis on automated identification of user behavior, such as emotional expressions, based on online social media texts. However, the majority of the prior attempts are based on traditional procedures that are insufficient to provide promising outcomes. In this study, we categorize emotional sentiments by recognizing them in the text. For that purpose, we present a deep learning model, bidirectional long-term short-term memory (BiLSMT), for emotion recognition that takes into account five main emotions (Joy, Sadness, Fear, Shame, Guilt). We use our experimental assessments on the emotion dataset to accomplish the emotion categorization job. The datasets were evaluated and the findings revealed that, when compared to state-of-the-art methodologies, the proposed model can successfully categorize user emotions into several classifications. Finally, we assess the efficacy of our strategy using statistical analysis. This research’s findings help firms to apply best practices in the selection, management, and optimization of policies, services, and product information.

Download Full-text

Towards a Reliable and Sensitive Deep Learning Based Approach for Multi-Ship Detection in SAR Imagery

International Journal of Geoinformatics ◽

10.52939/ijg.v17i6.2067 ◽

2021 ◽

pp. 59-70

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Relative Sensitivity ◽

Detection Techniques ◽

Ship Detection ◽

Detection Process ◽

Proposed Model ◽

Sar Imagery ◽

Comparative Results ◽

Deep Learning Model

Synthetic Aperture Radar (SAR) images show promising results in monitoring maritime activities. Recently, Deep learning-based object detection techniques have impressive results in most detection applications but unfortunately there are challenging problems such as difficulty of detecting multiple ships, especially inshore ones. In this paper, a three-step ship detection process is described and a reliable and sensitive hybrid deep learning model is proposed as an efficient classifier in the middle step. The proposed model combines the finetuned Inception-Resnet-V2 model and the Long Short Term Memory model in two different approaches: parallel approach and cascaded approach. In experiments, the region proposal algorithm and the Non-Maxima suppression algorithm are applied in the first and last step in the three-step detection process. The comparative results show that the proposed approach in cascaded form outperforms the competitive recent state-of-the-art approaches by enhancement up to 16.3%, 16.5%, and 18.9% in terms of recall, precision and mean average precision, respectively. Moreover, the proposed approach shows high relative sensitivity for challenged cases of both inshore and offshore scenes by enhancement ratios up to 81.88% and 24.58%, respectively in recall perspective.

Download Full-text

Regularized Deep LSTM Autoencoder for Phonological Deviation Assessment

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001421520029 ◽

2020 ◽

pp. 2152002

Author(s):

Sheena Christabel Pravin ◽

M. Palanivelan

Keyword(s):

Short Term Memory ◽

Drop Out ◽

Severity Assessment ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Phonological Features ◽

Decoder Architecture ◽

Long Short Term Memory ◽

Deep Learning Model

In this paper, the Deep Long-short term memory Autoencoder (DLAE), a regularized deep learning model, is proposed for the automatic severity assessment of phonological deviations which are crucial stuttering markers in children. This automatic noninvasive severity assessment plays a paramount role in prevenient diagnosis, progress inference, and post-care for the patients with specific speech disorder. The proposed model is an implementation of a multi-layered Autoencoder in the Encoder–Decoder architecture of the Long-Short Term Memory (LSTM) model with hierarchically appended hidden layers and hidden units. The DLAE has definite advantage over the baseline Autoencoders. During the training phase, the proposed DLAE reconstructs the phonological features in an unsupervised fashion and the latent bottleneck features are extracted from the Encoder. The trained and regularized DLAE model with drop out is then used to predict the severity of the phonological deviation with high precision and classification accuracy compared to the baseline models.

Download Full-text

Real-Time Prediction of Reliability of Dynamic Positioning Sub-Systems for Computation of Dynamic Positioning Reliability Index (DP-RI) Using Long Short Term Memory (LSTM)

Volume 1: Offshore Technology ◽

10.1115/omae2020-18844 ◽

2020 ◽

Author(s):

Charles Fernandez ◽

Shashi Bhushan Kumar ◽

Wai Lok Woo ◽

Rosemary Norman ◽

Arun Kr. Dev

Keyword(s):

Reliability Index ◽

Short Term Memory ◽

Mathematical Formulation ◽

Model Performance ◽

Data Availability ◽

Dynamic Positioning ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory

Abstract In this study, a framework using Long Short Term Memory (LSTM) for prediction of reliability of Dynamic Positioning (DP) sub-systems for computation of Dynamic Positioning Reliability Index (DP-RI) has been proposed. The DP System is complex with significant levels of integration between many sub-systems such as the Reference System, DP Control System, Thruster / Propulsion System, Power System, Electrical System and the Environment System to perform diverse control functions. The proposed framework includes a mathematical computation approach to compute reliability of DP sub-systems and a data driven approach to predict the reliability at a sub-system level for evaluation of model performance and accuracy. The framework results demonstrate excellent performance under a wide range of data availability and guaranteed lower computational burden for real-time non-linear optimization. There are three main components of the proposed architecture for the mathematical formulation of the DP sub-systems based on individual sensor arrangements within the sub-system, computation of reliability of sub-systems and optimized LSTM deep learning algorithm for prediction of its reliability. Firstly, the mathematical formulation for the reliability of sub-systems is determined based on the series/parallel arrangement of the sensors of each individual equipment item within the sub-systems. Secondly, the computation of the reliability of sub-systems is achieved through an integrated approach during complex operation of the vessel. Thirdly, the novel optimized LSTM network is constructed to predict the reliability of the subsystems while minimizing integral errors in the algorithm. In this paper, numerical simulations are set-up using a state-of-the-art advisory decision-making tool with mock-up and real-world data to give insights into the model performance and validate it against the existing risk assessment methodologies. Furthermore, we have analyzed the efficiency and stability of the proposed model against various levels of data availability. In conclusion the prediction accuracy of the proposed model is scalable and higher when compared with other model results.

Download Full-text