Evaluating Temporal Bias in Time Series Event Detection Methods

The detection of events in time series is an important task in several areas of knowledge where operations monitoring is essential. Experts often have to deal with choosing the most appropriate event detection method for a time series, which can be a complex task. There is a demand for benchmarking different methods in order to guide this choice. For this, standard classification accuracy metrics are usually adopted. However, they are insufficient for a qualitative analysis of the tendency of a method to precede or delay event detections. Such analysis is interesting for applications in which tolerance for "close" detections is important rather than focusing only on accurate ones. In this context, this paper proposes a more comprehensive event detection benchmark process, including an analysis of temporal bias of detection methods. For that, metrics based on the time distance between event detections and identified events (detection delay) are adopted. Computational experiments were conducted using real-world and synthetic datasets from Yahoo Labs and resources from the Harbinger framework for event detection. Adopting the proposed detection delay-based metrics helped obtain a complete overview of the performance and general behavior of detection methods.

Download Full-text

A multiple pattern complex event detection scheme based on decomposition and merge sharing for massive event streams

International Journal of Distributed Sensor Networks ◽

10.1177/1550147720961336 ◽

2020 ◽

Vol 16 (10) ◽

pp. 155014772096133

Author(s):

Jianhua Wang ◽

Bang Ji ◽

Feng Lin ◽

Shilei Lu ◽

Yubin Lan ◽

...

Keyword(s):

Event Detection ◽

Detection Method ◽

Detection Algorithm ◽

Detection Methods ◽

Detection Scheme ◽

Event Stream ◽

Detection Model ◽

Efficient Detection ◽

High Efficient ◽

Complex Events

Quickly detecting related primitive events for multiple complex events from massive event stream usually faces with a great challenge due to their single pattern characteristic of the existing complex event detection methods. Aiming to solve the problem, a multiple pattern complex event detection scheme based on decomposition and merge sharing is proposed in this article. The achievement of this article lies that we successfully use decomposition and merge sharing technology to realize the high-efficient detection for multiple complex events from massive event streams. Specially, in our scheme, we first use decomposition sharing technology to decompose pattern expressions into multiple subexpressions, which can provide many sharing opportunities for subexpressions. We then use merge sharing technology to construct a multiple pattern complex events by merging sharing all the same prefix, suffix, or subpattern into one based on the above decomposition results. As a result, our proposed detection method in this article can effectively solve the above problem. The experimental results show that the proposed detection method in this article outperforms some general detection methods in detection model and detection algorithm in multiple pattern complex event detection as a whole.

Download Full-text

Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Sensors ◽

10.3390/s20072131 ◽

2020 ◽

Vol 20 (7) ◽

pp. 2131 ◽

Cited By ~ 3

Author(s):

Affan Ahmed Toor ◽

Muhammad Usman ◽

Farah Younas ◽

Alvis Cheuk M. Fong ◽

Sajid Ali Khan ◽

...

Keyword(s):

Data Streams ◽

Detection Method ◽

Concept Drift ◽

Class Imbalance ◽

Health Data ◽

Smart Devices ◽

Detection Delay ◽

Medical Sensors ◽

Synthetic Datasets ◽

Almost All

With the increasing popularity of the Internet-of-Medical-Things (IoMT) and smart devices, huge volumes of data streams have been generated. This study aims to address the concept drift, which is a major challenge in the processing of voluminous data streams. Concept drift refers to overtime change in data distribution. It may occur in the medical domain, for example the medical sensors measuring for general healthcare or rehabilitation, which may switch their roles for ICU emergency operations when required. Detecting concept drifts becomes trickier when the class distributions in data are skewed, which is often true for medical sensors e-health data. Reactive Drift Detection Method (RDDM) is an efficient method for detecting long concepts. However, RDDM has a high error rate, and it does not handle class imbalance. We propose an Enhanced Reactive Drift Detection Method (ERDDM), which systematically generates strategies to handle concept drift with class imbalance in data streams. We conducted experiments to compare ERDDM with three contemporary techniques in terms of prediction error, drift detection delay, latency, and ability to handle data imbalance. The experimentation was done in Massive Online Analysis (MOA) on 48 synthetic datasets customized to possess the capabilities of data streams. ERDDM can handle abrupt and gradual drifts and performs better than all benchmarks in almost all experiments.

Download Full-text

Machine Committee Framework for Power Grid Disturbances Analysis Using Synchrophasors Data

Smart Cities ◽

10.3390/smartcities4010001 ◽

2020 ◽

Vol 4 (1) ◽

pp. 1-16

Author(s):

Haoran Niu ◽

Olufemi A. Omitaomu ◽

Qing C. Cao

Keyword(s):

Event Detection ◽

Change Point ◽

Power Grid ◽

Detection Algorithm ◽

Change Point Detection ◽

Detection Methods ◽

Detection Accuracy ◽

Disturbance Analysis ◽

Events Detection ◽

Point Detection

Events detection is a key challenge in power grid frequency disturbances analysis. Accurate detection of events is crucial for situational awareness of the power system. In this paper, we study the problem of events detection in power grid frequency disturbance analysis using synchrophasors data streams. Current events detection approaches for power grid rely on individual detection algorithm. This study integrates some of the existing detection algorithms using the concept of machine committee to develop improved detection approaches for grid disturbance analysis. Specifically, we propose two algorithms—an Event Detection Machine Committee (EDMC) algorithm and a Change-Point Detection Machine Committee (CPDMC) algorithm. Both algorithms use parallel architecture to fuse detection knowledge of its individual methods to arrive at an overall output. The EDMC algorithm combines five individual event detection methods, while the CPDMC algorithm combines two change-point detection methods. Each method performs the detection task separately. The overall output of each algorithm is then computed using a voting strategy. The proposed algorithms are evaluated using three case studies of actual power grid disturbances. Compared with the individual results of the various detection methods, we found that the EDMC algorithm is a better fit for analyzing synchrophasors data; it improves the detection accuracy; and it is suitable for practical scenarios.

Download Full-text

Using social media for sub-event detection during disasters

Journal Of Big Data ◽

10.1186/s40537-021-00467-1 ◽

2021 ◽

Vol 8 (1) ◽

Author(s):

Loris Belcastro ◽

Fabrizio Marozzo ◽

Domenico Talia ◽

Paolo Trunfio ◽

Francesco Branda ◽

...

Keyword(s):

Social Media ◽

Natural Disasters ◽

Event Detection ◽

High Accuracy ◽

New Method ◽

Catastrophic Events ◽

Social Media Platforms ◽

Synthetic Datasets ◽

Collapsed Buildings ◽

Events Detection

AbstractSocial media platforms have become fundamental tools for sharing information during natural disasters or catastrophic events. This paper presents SEDOM-DD (Sub-Events Detection on sOcial Media During Disasters), a new method that analyzes user posts to discover sub-events that occurred after a disaster (e.g., collapsed buildings, broken gas pipes, floods). SEDOM-DD has been evaluated with datasets of different sizes that contain real posts from social media related to different natural disasters (e.g., earthquakes, floods and hurricanes). Starting from such data, we generated synthetic datasets with different features, such as different percentages of relevant posts and/or geotagged posts. Experiments performed on both real and synthetic datasets showed that SEDOM-DD is able to identify sub-events with high accuracy. For example, with a percentage of relevant posts of 80% and geotagged posts of 15%, our method detects the sub-events and their areas with an accuracy of 85%, revealing the high accuracy and effectiveness of the proposed approach.

Download Full-text

Identification of Spikes in Time Series

Epidemiologic Methods ◽

10.1515/em-2018-0005 ◽

2019 ◽

Vol 8 (1) ◽

Cited By ~ 3

Author(s):

Dana E. Goin ◽

Jennifer Ahern

Keyword(s):

Time Series ◽

Sensitivity And Specificity ◽

Kalman Filtering ◽

Detection Method ◽

Detection Methods ◽

Spike Detection ◽

Overall Performance ◽

The Mean ◽

Arima Modeling ◽

Smoothing Procedure

Abstract Researchers interested in the effects of exposure spikes on an outcome need tools to identify unexpectedly high values in a time series. However, the best method to identify spikes in time series is not known. This paper aims to fill this gap by testing the performance of several spike detection methods in a simulation setting. We created simulations parameterized by monthly violence rates in nine California cities that represented different series features, and randomly inserted spikes into the series. We then compared the ability to detect spikes of the following methods: ARIMA modeling, Kalman filtering and smoothing, wavelet modeling with soft thresholding, and an iterative outlier detection method. We varied the magnitude of spikes from 10 to 50 % of the mean rate over the study period and varied the number of spikes inserted from 1 to 10. We assessed performance of each method using sensitivity and specificity. The Kalman filtering and smoothing procedure had the best overall performance. We applied each method to the monthly violence rates in nine California cities and identified spikes in the rate over the 2005–2012 period.

Download Full-text

Experimental Comparison and Survey of Twelve Time Series Anomaly Detection Algorithms

Journal of Artificial Intelligence Research ◽

10.1613/jair.1.12698 ◽

2021 ◽

Vol 72 ◽

pp. 849-899

Author(s):

Cynthia Freeman ◽

Jonathan Merriman ◽

Ian Beaver ◽

Abdullah Mueen

Keyword(s):

Time Series ◽

Anomaly Detection ◽

Detection Method ◽

Concept Drift ◽

Area Under The Curve ◽

Detection Methods ◽

Experimental Comparison ◽

Detection Algorithms ◽

Extensive Evaluation ◽

Online Setting

The existence of an anomaly detection method that is optimal for all domains is a myth. Thus, there exists a plethora of anomaly detection methods which increases every year for a wide variety of domains. But a strength can also be a weakness; given this massive library of methods, how can one select the best method for their application? Current literature is focused on creating new anomaly detection methods or large frameworks for experimenting with multiple methods at the same time. However, and especially as the literature continues to expand, an extensive evaluation of every anomaly detection method is simply not feasible. To reduce this evaluation burden, we present guidelines to intelligently choose the optimal anomaly detection methods based on the characteristics the time series displays such as seasonality, trend, level change concept drift, and missing time steps. We provide a comprehensive experimental validation and survey of twelve anomaly detection methods over different time series characteristics to form guidelines based on several metrics: the AUC (Area Under the Curve), windowed F-score, and Numenta Anomaly Benchmark (NAB) scoring model. Applying our methodologies can save time and effort by surfacing the most promising anomaly detection methods instead of experimenting extensively with a rapidly expanding library of anomaly detection methods, especially in an online setting.

Download Full-text

Adaptive Drift Detection Mechanism for Non-Stationary Data Stream

Journal of Information & Knowledge Management ◽

10.1142/s0219649221500088 ◽

2021 ◽

Vol 20 (01) ◽

pp. 2150008

Author(s):

Nalini Nagendhiran ◽

Lakshmanan Kuppusamy

Keyword(s):

Traffic Management ◽

Data Stream ◽

Detection Method ◽

Concept Drift ◽

False Positive Rate ◽

Detection Mechanism ◽

Detection Delay ◽

Positive Rate ◽

Real World Datasets ◽

Synthetic Datasets

Mining is a challenging and important task in a non-stationary data stream. It is used in financial sectors, web log analysis, sensor networks, network traffic management, etc. In this environment, data distribution may change overtime and is called concept drift. So, it is necessary to identify the changes and address them to keep the model relevant to the incoming data. Many researchers have used Drift Detection Method (DDM). However, DDM is very sensitive to detect gradual drift where the detection delay is high. In this paper, we propose Adaptive Drift Detection Method (ADDM) which improves the performance of the drift detection mechanism. The ADDM uses a new parameter to detect the gradual drift in order to reduce the detection delay. The proposed method, ADDM, experiments with six synthetic datasets and four real-world datasets. Experimental results confirm that ADDM reduces the drift detection delay and false-positive rate (FPR) while preserving high classification accuracy.

Download Full-text