Binary versus Multiclass Deep Learning Modelling in Energy Disaggregation

AbstractThis paper compares two different deep-learning architectures for the use in energy disaggregation and Non-Intrusive Load Monitoring. Non-Intrusive Load Monitoring breaks down the aggregated energy consumption into individual appliance consumptions, thus detecting device operation. In detail, the “One versus All” approach, where one deep neural network per appliance is trained, and the “Multi-Output” approach, where the number of output nodes is equal to the number of appliances, are compared to each other. Evaluation is done on a state-of-the-art baseline system using standard performance measures and a set of publicly available datasets out of the REDD database.

Download Full-text

Improving Non-Intrusive Load Disaggregation through an Attention-Based Deep Neural Network

Energies ◽

10.3390/en14040847 ◽

2021 ◽

Vol 14 (4) ◽

pp. 847

Author(s):

Veronica Piccialli ◽

Antonio M. Sudoso

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Attention Mechanism ◽

Power Demand ◽

Generalization Capability ◽

Experimental Conditions ◽

Energy Disaggregation ◽

Load Monitoring ◽

High Power Consumption ◽

The Individual

Energy disaggregation, known in the literature as Non-Intrusive Load Monitoring (NILM), is the task of inferring the power demand of the individual appliances given the aggregate power demand recorded by a single smart meter which monitors multiple appliances. In this paper, we propose a deep neural network that combines a regression subnetwork with a classification subnetwork for solving the NILM problem. Specifically, we improve the generalization capability of the overall architecture by including an encoder–decoder with a tailored attention mechanism in the regression subnetwork. The attention mechanism is inspired by the temporal attention that has been successfully applied in neural machine translation, text summarization, and speech recognition. The experiments conducted on two publicly available datasets—REDD and UK-DALE—show that our proposed deep neural network outperforms the state-of-the-art in all the considered experimental conditions. We also show that modeling attention translates into the network’s ability to correctly detect the turning on or off an appliance and to locate signal sections with high power consumption, which are of extreme interest in the field of energy disaggregation.

Download Full-text

Methods for Host-Based Intrusion Detection with Deep Learning

Digital Threats: Research and Practice ◽

10.1145/3461462 ◽

2021 ◽

Author(s):

John Ring ◽

Colin Van Oort ◽

Samson Durst ◽

Vanessa White ◽

Joseph Near ◽

...

Keyword(s):

Deep Learning ◽

Intrusion Detection ◽

Black Box ◽

Intrusion Detection Systems ◽

System Behavior ◽

Detection Systems ◽

System Calls ◽

Baseline System ◽

Sequence Behavior ◽

Learning Architectures

Host-based Intrusion Detection Systems (HIDS) automatically detect events that indicate compromise by adversarial applications. HIDS are generally formulated as analyses of sequences of system events such as bash commands or system calls. Anomaly-based approaches to HIDS leverage models of normal (aka baseline) system behavior to detect and report abnormal events, and have the advantage of being able to detect novel attacks. In this paper we develop a new method for anomaly-based HIDS using deep learning predictions of sequence-to-sequence behavior in system calls. Our proposed method, called the ALAD algorithm, aggregates predictions at the application level to detect anomalies. We investigate the use of several deep learning architectures, including WaveNet and several recurrent networks. We show that ALAD empowered with deep learning significantly outperforms previous approaches. We train and evaluate our models using an existing dataset, ADFA-LD, and a new dataset of our own construction, PLAID. As deep learning models are black box in nature we use an alternate approach, allotaxonographs, to characterize and understand differences in baseline vs.~attack sequences in HIDS datasets such as PLAID.

Download Full-text

A Novel Deep Neural Network Model for Image Classification

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.f8832.088619 ◽

2019 ◽

Vol 8 (6) ◽

pp. 3241-3249

Keyword(s):

Neural Network ◽

Deep Learning ◽

Deep Neural Network ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Trained Neural Network ◽

Learning Architectures ◽

Better Than

In this article, we have trained neural network based on deep learning architectures to classify images on standard Fashion-MNIST and CIFAR-10 dataset. The various CNN- based classification architecture and RNN-based classification architecture are trained as well as tested on those standard datasets. In CNN architecture, we include CNN with 1, 2 and 3 Convolutional Layer and in RNN architecture, we include Long- Short Term Memory (LSTM) with one and two LSTM layer. Our models show remarkable outcome on the standard benchmark dataset. The tested models like CNN1 show greater accuracy on the MNIST fashion dataset and CNN3, LSTM1 and LSTM2 performed better than other models on the CIFAR-10 dataset.

Download Full-text

Visual Speech In Real Noisy Environments (VISION): A Novel Benchmark Dataset and Deep Learning-Based Baseline System

10.21437/interspeech.2020-2935 ◽

2020 ◽

Author(s):

Mandar Gogate ◽

Kia Dashtipour ◽

Amir Hussain

Keyword(s):

Deep Learning ◽

Benchmark Dataset ◽

Visual Speech ◽

Noisy Environments ◽

Baseline System

Download Full-text

A Novel Deep Learning Model by Stacking Conditional Restricted Boltzmann Machine and Deep Neural Network

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining ◽

10.1145/3394486.3403184 ◽

2020 ◽

Author(s):

Tianyu Kang ◽

Ping Chen ◽

John Quackenbush ◽

Wei Ding

Keyword(s):

Neural Network ◽

Deep Learning ◽

Deep Neural Network ◽

Learning Model ◽

Restricted Boltzmann Machine ◽

Boltzmann Machine ◽

Deep Learning Model

Download Full-text

A Generalization Performance Study Using Deep Learning Networks in Embedded Systems

Sensors ◽

10.3390/s21041031 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1031

Author(s):

Joseba Gorospe ◽

Rubén Mulero ◽

Olatz Arbelaitz ◽

Javier Muguerza ◽

Miguel Ángel Antón

Keyword(s):

Deep Learning ◽

Embedded Systems ◽

Embedded System ◽

General Purpose ◽

Learning Networks ◽

Performance Study ◽

Learning Techniques ◽

Wide Range ◽

Learning Architectures

Deep learning techniques are being increasingly used in the scientific community as a consequence of the high computational capacity of current systems and the increase in the amount of data available as a result of the digitalisation of society in general and the industrial world in particular. In addition, the immersion of the field of edge computing, which focuses on integrating artificial intelligence as close as possible to the client, makes it possible to implement systems that act in real time without the need to transfer all of the data to centralised servers. The combination of these two concepts can lead to systems with the capacity to make correct decisions and act based on them immediately and in situ. Despite this, the low capacity of embedded systems greatly hinders this integration, so the possibility of being able to integrate them into a wide range of micro-controllers can be a great advantage. This paper contributes with the generation of an environment based on Mbed OS and TensorFlow Lite to be embedded in any general purpose embedded system, allowing the introduction of deep learning architectures. The experiments herein prove that the proposed system is competitive if compared to other commercial systems.

Download Full-text

Chest x-ray automated triage: a semiologic approach designed for clinical implementation, exploiting different types of labels through a combination of four Deep Learning architectures.

Computer Methods and Programs in Biomedicine ◽

10.1016/j.cmpb.2021.106130 ◽

2021 ◽

pp. 106130

Author(s):

Candelaria Mosquera ◽

Facundo Nahuel Diaz ◽

Fernando Binder ◽

José Martín Rabellino ◽

Sonia Elizabeth Benitez ◽

...

Keyword(s):

Deep Learning ◽

Clinical Implementation ◽

X Ray ◽

Different Types ◽

Chest X Ray ◽

Learning Architectures

Download Full-text

Efficient Design of Energy Disaggregation Model with BERT-NILM Trained by AdaX Optimization Method for Smart Grid

Energies ◽

10.3390/en14154649 ◽

2021 ◽

Vol 14 (15) ◽

pp. 4649

Author(s):

İsmail Hakkı ÇAVDAR ◽

Vahit FERYAD

Keyword(s):

Smart Grids ◽

Energy Demand ◽

Optimization Technique ◽

Cost Effective ◽

Optimization Method ◽

Long Term Memory ◽

Successful Implementation ◽

Efficient Design ◽

Energy Disaggregation ◽

Load Monitoring

One of the basic conditions for the successful implementation of energy demand-side management (EDM) in smart grids is the monitoring of different loads with an electrical load monitoring system. Energy and sustainability concerns present a multitude of issues that can be addressed using approaches of data mining and machine learning. However, resolving such problems due to the lack of publicly available datasets is cumbersome. In this study, we first designed an efficient energy disaggregation (ED) model and evaluated it on the basis of publicly available benchmark data from the Residential Energy Disaggregation Dataset (REDD), and then we aimed to advance ED research in smart grids using the Turkey Electrical Appliances Dataset (TEAD) containing household electricity usage data. In addition, the TEAD was evaluated using the proposed ED model tested with benchmark REDD data. The Internet of things (IoT) architecture with sensors and Node-Red software installations were established to collect data in the research. In the context of smart metering, a nonintrusive load monitoring (NILM) model was designed to classify household appliances according to TEAD data. A highly accurate supervised ED is introduced, which was designed to raise awareness to customers and generate feedback by demand without the need for smart sensors. It is also cost-effective, maintainable, and easy to install, it does not require much space, and it can be trained to monitor multiple devices. We propose an efficient BERT-NILM tuned by new adaptive gradient descent with exponential long-term memory (Adax), using a deep learning (DL) architecture based on bidirectional encoder representations from transformers (BERT). In this paper, an improved training function was designed specifically for tuning of NILM neural networks. We adapted the Adax optimization technique to the ED field and learned the sequence-to-sequence patterns. With the updated training function, BERT-NILM outperformed state-of-the-art adaptive moment estimation (Adam) optimization across various metrics on REDD datasets; lastly, we evaluated the TEAD dataset using BERT-NILM training.

Download Full-text

Multimodal Deep Learning and Visible-Light and Hyperspectral Imaging for Fruit Maturity Estimation

Sensors ◽

10.3390/s21041288 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1288

Author(s):

Cinmayii A. Garillos-Manliguez ◽

John Y. Chiang

Keyword(s):

Deep Learning ◽

Visible Light ◽

Hyperspectral Imaging ◽

Morphological Changes ◽

Consumer Preference ◽

Hyperspectral Data ◽

Sensitivity Analyses ◽

Deep Convolutional Neural Networks ◽

Fruit Maturity ◽

Learning Architectures

Fruit maturity is a critical factor in the supply chain, consumer preference, and agriculture industry. Most classification methods on fruit maturity identify only two classes: ripe and unripe, but this paper estimates six maturity stages of papaya fruit. Deep learning architectures have gained respect and brought breakthroughs in unimodal processing. This paper suggests a novel non-destructive and multimodal classification using deep convolutional neural networks that estimate fruit maturity by feature concatenation of data acquired from two imaging modes: visible-light and hyperspectral imaging systems. Morphological changes in the sample fruits can be easily measured with RGB images, while spectral signatures that provide high sensitivity and high correlation with the internal properties of fruits can be extracted from hyperspectral images with wavelength range in between 400 nm and 900 nm—factors that must be considered when building a model. This study further modified the architectures: AlexNet, VGG16, VGG19, ResNet50, ResNeXt50, MobileNet, and MobileNetV2 to utilize multimodal data cubes composed of RGB and hyperspectral data for sensitivity analyses. These multimodal variants can achieve up to 0.90 F1 scores and 1.45% top-2 error rate for the classification of six stages. Overall, taking advantage of multimodal input coupled with powerful deep convolutional neural network models can classify fruit maturity even at refined levels of six stages. This indicates that multimodal deep learning architectures and multimodal imaging have great potential for real-time in-field fruit maturity estimation that can help estimate optimal harvest time and other in-field industrial applications.

Download Full-text