scholarly journals Smart Helmet: Thresh-Learner–Online Machine Learning on Data Streams

2019 ◽  
Vol 9 (1S3) ◽  
pp. 466-473
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Real Life ◽  
General Purpose ◽  
Streaming Data ◽  
Series Data ◽  
Dynamic Threshold ◽  
Wide Range ◽  
Set Up

Today, with an enormous generation and availability of time series data and streaming data, there is an increasing need for an automatic analyzing architecture to get fast interpretations and results. One of the significant potentiality of streaming analytics is to train and model each stream with unsupervised Machine Learning (ML) algorithms to detect anomalous behaviors, fuzzy patterns, and accidents in real-time. If executed reliably, each anomaly detection can be highly valuable for the application. In this paper, we propose a dynamic threshold setting system denoted as Thresh-Learner, mainly for the Internet of Things (IoT) applications that require anomaly detection. The proposed model enables a wide range of real-life applications where there is a necessity to set up a dynamic threshold over the streaming data to avoid anomalies, accidents or sending alerts to distant monitoring stations. We took the major problem of anomalies and accidents in coal mines due to coal fires and explosions. This results in loss of life due to the lack of automated alarming systems. We propose Thresh-Learner, a general purpose implementation for setting dynamic thresholds. We illustrate it through the Smart Helmet for coal mine workers which seamlessly integrates monitoring, analyzing and dynamic thresholds using IoT and analysis on the cloud.

scholarly journals Production fault simulation and forecasting from time series data with machine learning in glove textile industry

2019 ◽  
Vol 14 ◽  
pp. 155892501988346 ◽  
Author(s):  
Mine Seçkin ◽  
Ahmet Çağdaş Seçkin ◽  
Aysun Coşkun
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Production Process ◽  
Time Series Data ◽  
Real Life ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Random Parameters ◽  
Data Set ◽  
Textile Sector

Although textile production is heavily automation-based, it is viewed as a virgin area with regard to Industry 4.0. When the developments are integrated into the textile sector, efficiency is expected to increase. When data mining and machine learning studies are examined in textile sector, it is seen that there is a lack of data sharing related to production process in enterprises because of commercial concerns and confidentiality. In this study, a method is presented about how to simulate a production process and how to make regression from the time series data with machine learning. The simulation has been prepared for the annual production plan, and the corresponding faults based on the information received from textile glove enterprise and production data have been obtained. Data set has been applied to various machine learning methods within the scope of supervised learning to compare the learning performances. The errors that occur in the production process have been created using random parameters in the simulation. In order to verify the hypothesis that the errors may be forecast, various machine learning algorithms have been trained using data set in the form of time series. The variable showing the number of faulty products could be forecast very successfully. When forecasting the faulty product parameter, the random forest algorithm has demonstrated the highest success. As these error values have given high accuracy even in a simulation that works with uniformly distributed random parameters, highly accurate forecasts can be made in real-life applications as well.

scholarly journals Anomaly Detection of Pedestrian Flow: A Machine Learning Method for Monitoring-Data of Visitors to a Building

2020 ◽  
Vol 5 ◽  
Author(s):  
Kentaro Kumagai
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Emergency Evacuation ◽  
Integrated System ◽  
Series Data ◽  
Machine Learning Method ◽  
Pedestrian Flow ◽  
Infrared Sensors ◽  
Public Facilities

Many public facilities such as community halls and gymnasiums are supposed to be evacuation sites when disasters occur. From the viewpoint of managing such facilities, it is necessary to monitor the usage and to respond immediately when an anomaly occurs. In this study, an integrated system of IoT sensors and machine learning for anomaly detection of pedestrian flow was proposed for buildings that are expected to be used as emergency evacuation sites in the event of a disaster. For trial practice of the system, infrared sensors were installed in a research building of a university, and data of visitors to the fourth floor of the building was collected as a time series data of pedestrian flow. As a result, it was shown that anomalies of pedestrian flow at an arbitrary time of a day with an occurrence probability of 5 % or less can be detected properly using the data collected.

scholarly journals Location Prediction Models using Data Mining and Machine Learning

2020 ◽  
Vol 9 (3) ◽  
pp. 3383-3390
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Time Series Data ◽  
Real Life ◽  
Series Data ◽  
Data Sets ◽  
Location Prediction ◽  
Summary Table ◽  
User Data ◽  
Using Data

A vast availability of location based user data which is generated everyday whether it is GPS data from online cabs, or weather time series data, is essential in many ways to the user and has been applied to many real life applications such as location targeted-advertising, recommendation systems, crime-rate detection, home trajectory analysis etc. In order to analyze this data and use it to fruitfulness a vast majority of prediction models have been proposed and utilized over the years. A next location prediction model is a model that uses this data and can be designed as a combination of two or more models and techniques, but these have their own pros and cons. The aim of this document is to analyze and compare the various machine learning models and related experiments that can be applied for better location prediction algorithms in the near future. The paper is organized in a way so as to give readers insights and other noteworthy points and inferences from the papers surveyed. A summary table has been presented to get a glimpse of the methods in depth and our added inferences along with the data-sets analyzed.

scholarly journals Self-Diagnosis of Multiphase Flow Meters through Machine Learning-Based Anomaly Detection

Energies ◽  
2020 ◽  
Vol 13 (12) ◽  
pp. 3136
Author(s):  
Tommaso Barbariol ◽  
Enrico Feltresi ◽  
Gian Antonio Susto
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Multiphase Flow ◽  
Time Series Data ◽  
Synthetic Data ◽  
Machine Learning Algorithms ◽  
Gas Content ◽  
Series Data ◽  
Oil Well ◽  
Flow Meters

Measuring systems are becoming increasingly sophisticated in order to tackle the challenges of modern industrial problems. In particular, the Multiphase Flow Meter (MPFM) combines different sensors and data fusion techniques to estimate quantities that are difficult to be measured like the water or gas content of a multiphase flow, coming from an oil well. The evaluation of the flow composition is essential for the well productivity prediction and management, and for this reason, the quantification of the meter measurement quality is crucial. While instrument complexity is increasing, demands for confidence levels in the provided measures are becoming increasingly more common. In this work, we propose an Anomaly Detection approach, based on unsupervised Machine Learning algorithms, that enables the metrology system to detect outliers and to provide a statistical level of confidence in the measures. The proposed approach, called AD4MPFM (Anomaly Detection for Multiphase Flow Meters), is designed for embedded implementation and for multivariate time-series data streams. The approach is validated both on real and synthetic data.

scholarly journals Neural Activity Classification with Machine Learning Models Trained on Interspike Interval Time-Series Data

2021 ◽  
Author(s):  
Ivan Lazarevich ◽  
Ilya Prokin ◽  
Boris Gutkin ◽  
Victor Kazantsev
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Language Processing ◽  
Neural Activity ◽  
Time Series Data ◽  
Series Data ◽  
Neural Decoding ◽  
Learning Models ◽  
Wide Range ◽  
Machine Learning Models

Modern well-performing approaches to neural decoding are based on machine learning models such as decision tree ensembles and deep neural networks. The wide range of algorithms that can be utilized to learn from neural spike trains, which are essentially time-series data, results in the need for diverse and challenging benchmarks for neural decoding, similar to the ones in the fields of computer vision and natural language processing. In this work, we propose a spike train classification benchmark, based on open-access neural activity datasets and consisting of several learning tasks such as stimulus type classification, animal’s behavioral state prediction and neuron type identification. We demonstrate that an approach based on hand-crafted time-series feature engineering establishes a strong baseline performing on par with state-of-the-art deep learning based models for neural decoding. We release the code allowing to reproduce the reported results 1.

scholarly journals Anomaly Detection with Machine Learning Algorithms and Big Data in Electricity Consumption

2021 ◽  
Vol 13 (19) ◽  
pp. 10963
Author(s):  
Simona-Vasilica Oprea ◽  
Adela Bâra ◽  
Florina Camelia Puican ◽  
Ioan Cosmin Radu
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Hybrid Approach ◽  
Electricity Consumption ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Smart Meters ◽  
Linear Discriminant

When analyzing smart metering data, both reading errors and frauds can be identified. The purpose of this analysis is to alert the utility companies to suspicious consumption behavior that could be further investigated with on-site inspections or other methods. The use of Machine Learning (ML) algorithms to analyze consumption readings can lead to the identification of malfunctions, cyberattacks interrupting measurements, or physical tampering with smart meters. Fraud detection is one of the classical anomaly detection examples, as it is not easy to label consumption or transactional data. Furthermore, frauds differ in nature, and learning is not always possible. In this paper, we analyze large datasets of readings provided by smart meters installed in a trial study in Ireland by applying a hybrid approach. More precisely, we propose an unsupervised ML technique to detect anomalous values in the time series, establish a threshold for the percentage of anomalous readings from the total readings, and then label that time series as suspicious or not. Initially, we propose two types of algorithms for anomaly detection for unlabeled data: Spectral Residual-Convolutional Neural Network (SR-CNN) and an anomaly trained model based on martingales for determining variations in time-series data streams. Then, the Two-Class Boosted Decision Tree and Fisher Linear Discriminant analysis are applied on the previously processed dataset. By training the model, we obtain the required capabilities of detecting suspicious consumers proved by an accuracy of 90%, precision score of 0.875, and F1 score of 0.894.

scholarly journals Performance Comparison of Anomaly Detection Algorithms for Streaming Data

2020 ◽  
Author(s):  
Zirije Hasani ◽  
Jakup Fondaj
Keyword(s):  
Anomaly Detection ◽  
Real Time ◽  
Time Series Data ◽  
Scale Up ◽  
Detection Algorithm ◽  
Streaming Data ◽  
Series Data ◽  
Process Data ◽  
Significant Information ◽  
Detection Algorithms

Abstract Most of the today's world data are streaming, time-series data, where anomalies detection gives significant information of possible critical situations. Yet, detecting anomalies in big streaming data is a difficult task, requiring detectors to acquire and process data in a real-time, as they occur, even before they are stored and instantly alarm on potential threats. Suitable to the need for real-time alarm and unsupervised procedures for massive streaming data anomaly detection, algorithms have to be robust, with low processing time, eventually at the cost of the accuracy. In this work we compare the performance of our proposed anomaly detection algorithm HW-GA[1] with other existing methods as ARIMA [10], Moving Average [11] and Holt Winters [12]. The algorithms are tested and results are visualized in the system R, on the three Numenta datasets, with known anomalies and own e-dnevnik dataset with unknown anomalies. Evaluation is done by comparing achieved results (the algorithm execution time and CPU usage). Our interest is monitoring of the streaming log data that are generating in the national educational network (e-dnevnik) that acquires a massive number of online queries and to detect anomalies in order to scale up performance, prevent network downs, alarm on possible attacks and similar.

Detecting Interesting and Anomalous Patterns In Multivariate Time-Series Data in an Offshore Platform Using Unsupervised Learning

2021 ◽  
Author(s):  
Ilan Sousa Figueirêdo ◽  
Tássio Farias Carvalho ◽  
Wenisten José Dantas Silva ◽  
Lílian Lefol Nani Guarieiro ◽  
Erick Giovani Sperandio Nascimento
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Anomaly Detection ◽  
Unsupervised Learning ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Unsupervised Machine Learning

Abstract Detection of anomalous events in practical operation of oil and gas (O&G) wells and lines can help to avoid production losses, environmental disasters, and human fatalities, besides decreasing maintenance costs. Supervised machine learning algorithms have been successful to detect, diagnose, and forecast anomalous events in O&G industry. Nevertheless, these algorithms need a large quantity of annotated dataset and labelling data in real world scenarios is typically unfeasible because of exhaustive work of experts. Therefore, as unsupervised machine learning does not require an annotated dataset, this paper intends to perform a comparative evaluation performance of unsupervised learning algorithms to support experts for anomaly detection and pattern recognition in multivariate time-series data. So, the goal is to allow experts to analyze a small set of patterns and label them, instead of analyzing large datasets. This paper used the public 3W database of three offshore naturally flowing wells. The experiment used real data of production of O&G from underground reservoirs with the following anomalous events: (i) spurious closure of Downhole Safety Valve (DHSV) and (ii) quick restriction in Production Choke (PCK). Six unsupervised machine learning algorithms were assessed: Cluster-based Algorithm for Anomaly Detection in Time Series Using Mahalanobis Distance (C-AMDATS), Luminol Bitmap, SAX-REPEAT, k-NN, Bootstrap, and Robust Random Cut Forest (RRCF). The comparison evaluation of unsupervised learning algorithms was performed using a set of metrics: accuracy (ACC), precision (PR), recall (REC), specificity (SP), F1-Score (F1), Area Under the Receiver Operating Characteristic Curve (AUC-ROC), and Area Under the Precision-Recall Curve (AUC-PRC). The experiments only used the data labels for assessment purposes. The results revealed that unsupervised learning successfully detected the patterns of interest in multivariate data without prior annotation, with emphasis on the C-AMDATS algorithm. Thus, unsupervised learning can leverage supervised models through the support given to data annotation.

Exathlon

2021 ◽  
Vol 14 (11) ◽  
pp. 2613-2626
Author(s):  
Vincent Jacob ◽  
Fei Song ◽  
Arnaud Stiegler ◽  
Bijan Rad ◽  
Yanlei Diao ◽  
...  
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Data Science ◽  
Time Series Data ◽  
State Of The Art ◽  
Evaluation Methodology ◽  
Quality Data ◽  
Series Data ◽  
Data Repositories ◽  
Wide Range

Access to high-quality data repositories and benchmarks have been instrumental in advancing the state of the art in many experimental research domains. While advanced analytics tasks over time series data have been gaining lots of attention, lack of such community resources severely limits scientific progress. In this paper, we present Exathlon, the first comprehensive public benchmark for explainable anomaly detection over high-dimensional time series data. Exathlon has been systematically constructed based on real data traces from repeated executions of large-scale stream processing jobs on an Apache Spark cluster. Some of these executions were intentionally disturbed by introducing instances of six different types of anomalous events (e.g., misbehaving inputs, resource contention, process failures). For each of the anomaly instances, ground truth labels for the root cause interval as well as those for the extended effect interval are provided, supporting the development and evaluation of a wide range of anomaly detection (AD) and explanation discovery (ED) tasks. We demonstrate the practical utility of Exathlon's dataset, evaluation methodology, and end-to-end data science pipeline design through an experimental study with three state-of-the-art AD and ED techniques.

scholarly journals Towards Machine Learning-based Anomaly Detection on Time-Series Data

2021 ◽  
Vol 13 (1) ◽  
pp. 35-44
Author(s):  
Daniel Vajda ◽  
Adrian Pekar ◽  
Karoly Farkas
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Detection Methods ◽  
Series Data ◽  
Rich Information

The complexity of network infrastructures is exponentially growing. Real-time monitoring of these infrastructures is essential to secure their reliable operation. The concept of telemetry has been introduced in recent years to foster this process by streaming time-series data that contain feature-rich information concerning the state of network components. In this paper, we focus on a particular application of telemetry — anomaly detection on time-series data. We rigorously examined state-of-the-art anomaly detection methods. Upon close inspection of the methods, we observed that none of them suits our requirements as they typically face several limitations when applied on time-series data. This paper presents Alter-Re2, an improved version of ReRe, a state-of-the-art Long Short- Term Memory-based machine learning algorithm. Throughout a systematic examination, we demonstrate that by introducing the concepts of ageing and sliding window, the major limitations of ReRe can be overcome. We assessed the efficacy of Alter-Re2 using ten different datasets and achieved promising results. Alter-Re2 performs three times better on average when compared to ReRe.

Sign in / Sign up

Export Citation Format

Share Document