Anomaly Detection on Data Streams for Smart Agriculture

Smart agriculture technologies are effective instruments for increasing farm sustainability and production. They generate many spatial, temporal, and time-series data streams that, when analysed, can reveal several issues on farm productivity and efficiency. In this context, the detection of anomalies can help in the identification of observations that deviate from the norm. This paper proposes an adaptation of an ensemble anomaly detector called enhanced locally selective combination in parallel outlier ensembles (ELSCP). On this basis, we define an unsupervised data-driven methodology for smart-farming temporal data that is applied in two case studies. The first considers harvest data including combine-harvester Global Positioning System (GPS) traces. The second is dedicated to crop data where we study the link between crop state (damaged or not) and detected anomalies. Our experiments show that our methodology achieved interesting performance with Area Under the Curve of Precision-Recall (AUCPR) score of 0.972 in the combine-harvester dataset, which is 58.7% better than that of the second-best approach. In the crop dataset, our analysis showed that 30% of the detected anomalies could be directly linked to crop damage. Therefore, anomaly detection could be integrated in the decision process of farm operators to improve harvesting efficiency and crop health.

Download Full-text

An Anomaly Detection Method with Exemplar Subsequence for Time Series Data

IEEJ Transactions on Electronics Information and Systems ◽

10.1541/ieejeiss.136.363 ◽

2016 ◽

Vol 136 (3) ◽

pp. 363-372

Author(s):

Takaaki Nakamura ◽

Makoto Imamura ◽

Masashi Tatedoko ◽

Norio Hirai

Keyword(s):

Time Series ◽

Anomaly Detection ◽

Time Series Data ◽

Detection Method ◽

Series Data

Download Full-text

LRZ Convolution: An Algorithm for Automatic Anomaly Detection in Time-series Data

32nd International Conference on Scientific and Statistical Database Management ◽

10.1145/3400903.3400904 ◽

2020 ◽

Author(s):

Arunprasad P. Marathe

Keyword(s):

Time Series ◽

Anomaly Detection ◽

Time Series Data ◽

Series Data

Download Full-text

Indirectly Supervised Anomaly Detection of Clinically Meaningful Health Events from Smart Home Data

ACM Transactions on Intelligent Systems and Technology ◽

10.1145/3439870 ◽

2021 ◽

Vol 12 (2) ◽

pp. 1-18

Author(s):

Jessamyn Dahmen ◽

Diane J. Cook

Keyword(s):

Anomaly Detection ◽

Time Series Data ◽

Bayesian Optimization ◽

Detection Methods ◽

Series Data ◽

Detection Techniques ◽

Health Events ◽

Warm Start ◽

Health Related ◽

Unsupervised Algorithms

Anomaly detection techniques can extract a wealth of information about unusual events. Unfortunately, these methods yield an abundance of findings that are not of interest, obscuring relevant anomalies. In this work, we improve upon traditional anomaly detection methods by introducing Isudra, an Indirectly Supervised Detector of Relevant Anomalies from time series data. Isudra employs Bayesian optimization to select time scales, features, base detector algorithms, and algorithm hyperparameters that increase true positive and decrease false positive detection. This optimization is driven by a small amount of example anomalies, driving an indirectly supervised approach to anomaly detection. Additionally, we enhance the approach by introducing a warm-start method that reduces optimization time between similar problems. We validate the feasibility of Isudra to detect clinically relevant behavior anomalies from over 2M sensor readings collected in five smart homes, reflecting 26 health events. Results indicate that indirectly supervised anomaly detection outperforms both supervised and unsupervised algorithms at detecting instances of health-related anomalies such as falls, nocturia, depression, and weakness.

Download Full-text

Change Point Enhanced Anomaly Detection for IoT Time Series Data

Water ◽

10.3390/w13121633 ◽

2021 ◽

Vol 13 (12) ◽

pp. 1633

Author(s):

Elena-Simona Apostol ◽

Ciprian-Octavian Truică ◽

Florin Pop ◽

Christian Esposito

Keyword(s):

Time Series ◽

Anomaly Detection ◽

Change Point ◽

Time Series Data ◽

Multivariate Time Series ◽

Change Point Detection ◽

Change Points ◽

Series Data ◽

Prediction And Forecasting ◽

Point Detection

Due to the exponential growth of the Internet of Things networks and the massive amount of time series data collected from these networks, it is essential to apply efficient methods for Big Data analysis in order to extract meaningful information and statistics. Anomaly detection is an important part of time series analysis, improving the quality of further analysis, such as prediction and forecasting. Thus, detecting sudden change points with normal behavior and using them to discriminate between abnormal behavior, i.e., outliers, is a crucial step used to minimize the false positive rate and to build accurate machine learning models for prediction and forecasting. In this paper, we propose a rule-based decision system that enhances anomaly detection in multivariate time series using change point detection. Our architecture uses a pipeline that automatically manages to detect real anomalies and remove the false positives introduced by change points. We employ both traditional and deep learning unsupervised algorithms, in total, five anomaly detection and five change point detection algorithms. Additionally, we propose a new confidence metric based on the support for a time series point to be an anomaly and the support for the same point to be a change point. In our experiments, we use a large real-world dataset containing multivariate time series about water consumption collected from smart meters. As an evaluation metric, we use Mean Absolute Error (MAE). The low MAE values show that the algorithms accurately determine anomalies and change points. The experimental results strengthen our assumption that anomaly detection can be improved by determining and removing change points as well as validates the correctness of our proposed rules in real-world scenarios. Furthermore, the proposed rule-based decision support systems enable users to make informed decisions regarding the status of the water distribution network and perform effectively predictive and proactive maintenance.

Download Full-text