scholarly journals Detection of abrupt shift and non-parametric analyses of trends in runoff time series in Dez River Basin

2021 ◽  
Author(s):  
Hooman Gholami ◽  
Yahya Moradi ◽  
Morteza Lotfirad ◽  
Mohammad Amin Gandomi ◽  
Nooredin Bazgir ◽  
...  
Keyword(s):  
Time Series ◽  
Change Point ◽  
Change Points ◽  
Limit Values ◽  
June July August ◽  
Level Of Confidence ◽  
Observatory Data ◽  
Parametric Analyses ◽  
June July ◽  
Hydrometric Station

Abstract The present study aims to investigate the homogeneity of runoff time series and also to review the existence of trends in Tale Zang hydrometric station (the hydrometric station in the entrance of Dez Reservoir) runoff, using 61 years (1956–2016) daily observatory data. Pettit test, which is a common method in investigating the homogeneity of time series, was used to identify change points. Both Mann-Kendall and auto-correlated Mann-Kendall tests were applied to analyze the existence or non-existence of trends in each annual, seasonal, and monthly time series observed in a runoff. In time series, significant trends in 95% level of confidence were recognized, upper and lower limit values were presented for Sen's slope and it was tested for the increasing or decreasing trends in nature. Based on the results of this study, the significant change point in 95% level of confidence was recognized in Annual, Spring, Summer, Autumn, March, May, June, July, August, September, and October in 1997,1997, 1999, 1997, 1999, 1999, 1999, 1997, 2000, 2007, 2008, respectively. Analyzing the existence of a trend in 95% level of confidence indicated that in Spring, and in March, September, and October, for both Mann-Kendall and auto-correlated Mann-Kendall tests, the trend is significant and additive until the change point.

scholarly journals Change Point Enhanced Anomaly Detection for IoT Time Series Data

Water ◽  
2021 ◽  
Vol 13 (12) ◽  
pp. 1633
Author(s):  
Elena-Simona Apostol ◽  
Ciprian-Octavian Truică ◽  
Florin Pop ◽  
Christian Esposito
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Change Point ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Change Point Detection ◽  
Change Points ◽  
Series Data ◽  
Prediction And Forecasting ◽  
Point Detection

Due to the exponential growth of the Internet of Things networks and the massive amount of time series data collected from these networks, it is essential to apply efficient methods for Big Data analysis in order to extract meaningful information and statistics. Anomaly detection is an important part of time series analysis, improving the quality of further analysis, such as prediction and forecasting. Thus, detecting sudden change points with normal behavior and using them to discriminate between abnormal behavior, i.e., outliers, is a crucial step used to minimize the false positive rate and to build accurate machine learning models for prediction and forecasting. In this paper, we propose a rule-based decision system that enhances anomaly detection in multivariate time series using change point detection. Our architecture uses a pipeline that automatically manages to detect real anomalies and remove the false positives introduced by change points. We employ both traditional and deep learning unsupervised algorithms, in total, five anomaly detection and five change point detection algorithms. Additionally, we propose a new confidence metric based on the support for a time series point to be an anomaly and the support for the same point to be a change point. In our experiments, we use a large real-world dataset containing multivariate time series about water consumption collected from smart meters. As an evaluation metric, we use Mean Absolute Error (MAE). The low MAE values show that the algorithms accurately determine anomalies and change points. The experimental results strengthen our assumption that anomaly detection can be improved by determining and removing change points as well as validates the correctness of our proposed rules in real-world scenarios. Furthermore, the proposed rule-based decision support systems enable users to make informed decisions regarding the status of the water distribution network and perform effectively predictive and proactive maintenance.

scholarly journals Closed-Form Estimation of Multiple Change-Point Models

2013 ◽  
Author(s):  
Greg Jensen
Keyword(s):  
Time Series ◽  
Linear Regression ◽  
Change Point ◽  
Marginal Likelihood ◽  
Stationary Time Series ◽  
Change Points ◽  
Model Complexity ◽  
Marginal Model ◽  
General Strategy ◽  
Change Point Models

Identifying discontinuities (or change-points) in otherwise stationary time series is a powerful analytic tool. This paper outlines a general strategy for identifying an unknown number of change-points using elementary principles of Bayesian statistics. Using a strategy of binary partitioning by marginal likelihood, a time series is recursively subdivided on the basis of whether adding divisions (and thus increasing model complexity) yields a justified improvement in the marginal model likelihood. When this approach is combined with the use of conjugate priors, it yields the Conjugate Partitioned Recursion (CPR) algorithm, which identifies change-points without computationally intensive numerical integration. Using the CPR algorithm, methods are described for specifying change-point models drawn from a host of familiar distributions, both discrete (binomial, geometric, Poisson) and continuous (exponential, Gaussian, uniform, and multiple linear regression), as well as multivariate distribution (multinomial, multivariate normal, and multivariate linear regression). Methods by which the CPR algorithm could be extended or modified are discussed, and several detailed applications to data published in psychology and biomedical engineering are described.

Ecological change points: The strength of density dependence and the loss of history

2017 ◽  
Author(s):  
José M. Ponciano ◽  
Mark L. Taper ◽  
Brian Dennis
Keyword(s):  
Time Series ◽  
Population Dynamics ◽  
Density Dependence ◽  
Change Point ◽  
Likelihood Estimation ◽  
Theory And Practice ◽  
Change Points ◽  
Ecological Change ◽  
Proposed Model ◽  
Change Point Model

AbstractChange points in the dynamics of animal abundances have extensively been recorded in historical time series records. Little attention has been paid to the theoretical dynamic consequences of such change-points. Here we propose a change-point model of stochastic population dynamics. This investigation embodies a shift of attention from the problem of detecting when a change will occur, to another non-trivial puzzle: using ecological theory to understand and predict the post-breakpoint behavior of the population dynamics. The proposed model and the explicit expressions derived here predict and quantify how density dependence modulates the influence of the pre-breakpoint parameters into the post-breakpoint dynamics. Time series transitioning from one stationary distribution to another contain information about where the process was before the change-point, where is it heading and how long it will take to transition, and here this information is explicitly stated. Importantly, our results provide a direct connection of the strength of density dependence with theoretical properties of dynamic systems, such as the concept of resilience. Finally, we illustrate how to harness such information through maximum likelihood estimation for state-space models, and test the model robustness to widely different forms of compensatory dynamics. The model can be used to estimate important quantities in the theory and practice of population recovery.

scholarly journals Homogeneity tests and non-parametric analyses of tendencies in precipitation time series in Keszthely, Western Hungary

2019 ◽  
Vol 139 (3-4) ◽  
pp. 849-859 ◽  
Author(s):  
Tímea Kocsis ◽  
Ilona Kovács-Székely ◽  
Angéla Anda
Keyword(s):  
Time Series ◽  
Change Points ◽  
Annual Precipitation ◽  
Trend Test ◽  
Climate Data ◽  
Homogeneity Tests ◽  
Break Points ◽  
The Mean ◽  
Parametric Analyses ◽  
Precipitation Records

Abstract This study aims to investigate the precipitation trends in Keszthely (Western Hungary, Central Europe) through an examination of historical climate data covering the past almost one and a half centuries. Pettitt’s test for homogeneity was employed to detect change points in the time series of monthly, seasonal and annual precipitation records. Change points and monotonic trends were analysed separately in annual, seasonal and monthly time series of precipitation. While no break points could be detected in the annual precipitation series, a significant decreasing trend of 0.2–0.7 mm/year was highlighted statistically using the autocorrelated Mann-Kendall trend test. Significant change points were found in those time series in which significant tendencies had been detected in previous studies. These points fell in spring and winter for the seasonal series, and October for the monthly series. The question therefore arises of whether these trends are the result of a shift in the mean. The downward and upward shift in the mean in the case of spring and winter seasonal amounts, respectively, leads to a suspicion that changes in precipitation are also in progress in these seasons. The study concludes that homogeneity tests are of great importance in such analyses, because they may help to avoid false trend detections.

scholarly journals Locally adaptive change-point detection (LACPD) with applications to environmental changes

2021 ◽  
Author(s):  
Mehdi Moradi ◽  
Manuel Montesino-SanMartin ◽  
M. Dolores Ugarte ◽  
Ana F. Militino
Keyword(s):  
Time Series ◽  
Change Point ◽  
Error Probability ◽  
Type I Error ◽  
Change Point Detection ◽  
Alternative Methods ◽  
Change Points ◽  
Type I ◽  
Type I Error Probability ◽  
Point Detection

AbstractWe propose an adaptive-sliding-window approach (LACPD) for the problem of change-point detection in a set of time-ordered observations. The proposed method is combined with sub-sampling techniques to compensate for the lack of enough data near the time series’ tails. Through a simulation study, we analyse its behaviour in the presence of an early/middle/late change-point in the mean, and compare its performance with some of the frequently used and recently developed change-point detection methods in terms of power, type I error probability, area under the ROC curves (AUC), absolute bias, variance, and root-mean-square error (RMSE). We conclude that LACPD outperforms other methods by maintaining a low type I error probability. Unlike some other methods, the performance of LACPD does not depend on the time index of change-points, and it generally has lower bias than other alternative methods. Moreover, in terms of variance and RMSE, it outperforms other methods when change-points are close to the time series’ tails, whereas it shows a similar (sometimes slightly poorer) performance as other methods when change-points are close to the middle of time series. Finally, we apply our proposal to two sets of real data: the well-known example of annual flow of the Nile river in Awsan, Egypt, from 1871 to 1970, and a novel remote sensing data application consisting of a 34-year time-series of satellite images of the Normalised Difference Vegetation Index in Wadi As-Sirham valley, Saudi Arabia, from 1986 to 2019. We conclude that LACPD shows a good performance in detecting the presence of a change as well as the time and magnitude of change in real conditions.

scholarly journals Spatio-Temporal Trends of Monthly and Annual Precipitation in Aguascalientes, Mexico

Atmosphere ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 437
Author(s):  
Osías Ruiz-Alvarez ◽  
Vijay P. Singh ◽  
Juan Enciso-Medina ◽  
Ronald Ernesto Ontiveros-Capurata ◽  
Arturo Corrales-Suastegui
Keyword(s):  
Time Series ◽  
Annual Precipitation ◽  
Irrigated Agriculture ◽  
Monthly Precipitation ◽  
Precipitation Trend ◽  
Precipitation Time Series ◽  
June July August ◽  
Precipitation Decrease ◽  
Weather Stations ◽  
June July

The objective of this research was to analyze the temporal patterns of monthly and annual precipitation at 36 weather stations of Aguascalientes, Mexico. The precipitation trend was determined by the Mann–Kendall method and the rate of change with the Theil–Sen estimator. In total, 468 time series were analyzed, 432 out of them were monthly, and 36 were annual. Out of the total monthly precipitation time series, 42 series showed a statistically significant trend (p ≤ 0.05), from which 8/34 showed a statistically significant negative/positive trend. The statistically significant negative trends of monthly precipitation occurred in January, April, October, and December. These trends denoted more significant irrigation water use, higher water extractions from the aquifers in autumn–winter, more significant drought occurrence, low forest productivity, higher wildfire risk, and greater frost risk. The statistically significant positive trends occurred in May, June, July, August, and September; to a certain extent, these would contribute to the hydrology, agriculture, and ecosystem but also could provoke problems due to water excess. In some months, the annual precipitation variability and El Niño-Southern Oscillation (ENSO) were statistically correlated, so it could be established that in Aguascalientes, this phenomenon is one of the causes of the yearly precipitation variation. Out of the total annual precipitation time series, only nine series were statistically significant positive; eight out of them originated by the augments of monthly precipitation. Thirteen weather stations showed statistically significant trends in the total precipitation of the growing season (May, June, July, August, and September); these stations are located in regions of irrigated agriculture. The precipitation decrease in dry months can be mitigated using shorter cycle varieties with lower water consumption, irrigation methods with high efficiency, and repairing irrigation infrastructure. The precipitation increase in humid months can be used to store water and use it during the dry season, and its adverse effects can be palliated with the use of varieties resistant to root diseases and lodging. The results of this work will be beneficial in the management of agriculture, hydrology, and water resources of Aguascalientes and in neighboring arid regions affected by climate change.

Change Point Detection For Clustered Expression Data In Short Time Series.

2021 ◽  
Author(s):  
Miriam Sieg ◽  
Lina Katrin Sciesielski ◽  
Karin Kirschner ◽  
Jochen Kruppa
Keyword(s):  
Time Series ◽  
Confidence Intervals ◽  
Change Point ◽  
Mixed Model ◽  
Linear Mixed Model ◽  
Change Point Detection ◽  
Change Points ◽  
Point Detection ◽  
Over Time ◽  
Time Point

Abstract Background: In longitudinal studies, observations are made over time. Hence, the single observations at each time point are dependent, making them a repeated measurement. In this work, we explore a different, counterintuitive setting: At each developmental time point, a lethal observation is performed on the pregnant or nursing mother. Therefore, the single time points are independent. Furthermore, the observation in the offspring at each time point is correlated with each other because each litter consists of several (genetically linked) littermates. In addition, the observed time series is short from a statistical perspective as animal ethics prevent killing more mother mice than absolutely necessary, and murine development is short anyway. We solve these challenges by using multiple contrast tests and visualizing the change point by the use of confidence intervals.Results: We used linear mixed models to model the variability of the mother. The estimates from the linear mixed model are then used in multiple contrast tests.There are a variety of contrasts and intuitively, we would use the Changepoint method. However, it does not deliver satisfying results. Interestingly, we found two other contrasts, both capable of answering different research questions in change point detection: i) Should a single point with change direction be found, or ii) Should the overall progression be determined? The Sequen contrast answers the first, the McDermott the second. Confidence intervals deliver effect estimates for the strength of the potential change point. Therefore, the scientist can define a biologically relevant limit of change depending on the research question.Conclusion: We present a solution with effect estimates for short independent time series with observations nested at a given time point. Multiple contrast tests produce confidence intervals, which allow determining the position of change points or to visualize the expression course over time. We suggest to use McDermott’s method to determine if there is an overall significant change within the time frame, while Sequen is better in determining specific change points. In addition, we offer a short formula for the estimation of the maximal length of the time series.

A Novel Change-Point Detection Approach for Monitoring High-Dimensional Traffics in Distributed Systems

2014 ◽  
Vol 536-537 ◽  
pp. 499-511 ◽  
Author(s):  
Li Zhao ◽  
Qian Liu ◽  
Peng Du ◽  
Ge Fu ◽  
Wei Cao
Keyword(s):  
Time Series ◽  
Distributed Systems ◽  
Change Point ◽  
Principal Component ◽  
Change Point Detection ◽  
Change Points ◽  
Clustering Methods ◽  
Detection Approach ◽  
Time Series Segmentation ◽  
Point Detection

Change-point detection is the problem of finding abrupt changes in time-series. However, the meaningful changes are usually difficult to identify from the original massive traffics, due to high dimension and strong periodicity. In this paper, we propose a novel change-point detection approach, which simultaneously detects change points from all dimensions of the traffics with three steps. We first reduce the dimensions by the classical Principal Component Analysis (PCA), then we apply an extended time-series segmentation method to detect the nontrivial change times, finally we identify the responsible applications for the changes by F-test. We demonstrate through experiments on datasets collected from four distributed systems with 44 applications that the proposed approach can effectively detect the nontrivial change points from the multivariate and periodical traffics. Our approach is more appropriate for mining the nontrivial changes in traffic data comparing with other clustering methods, such as center-based Kmeans and density-based DBSCAN.

scholarly journals Using Graph Spectral to solve Change Point Detection Problems

2018 ◽  
Author(s):  
Luis Gustavo C. Uzai ◽  
André Y. Kashiwabara
Keyword(s):  
Time Series ◽  
Change Point ◽  
State Of The Art ◽  
The State ◽  
Change Point Detection ◽  
Change Points ◽  
Graph Spectrum ◽  
Point Detection ◽  
Over Time

Time series are sequence of values distributed over time. Analyzing time series is important in many areas including medical, financial, aerospace, commercial and entertainment. Change Point Detection is the problem of identifying changes in meaning or distribution of data in a time series. This article presents Spec, a new algorithm that uses the graph spectrum to detect change points. The Spec was evaluated using the UCR Archive which is a large da- tabase of different time series. Spec performance was compared to the PELT, ECP, EDM, and gSeg algorithms. The results showed that Spec achieved a better accuracy compared to the state of the art in some specific scenarios and as efficient as in most cases evaluated.

CPMAN: Change Point Detection Approach in Time Series Based on the Prediction of Multi-stage Attention Networks

2021 ◽  
Vol 30 (05) ◽  
pp. 2150026
Author(s):  
Haizhou Du ◽  
Ziyi Duan ◽  
Yang Zheng
Keyword(s):  
Time Series ◽  
Change Point ◽  
Early Stage ◽  
Change Point Detection ◽  
Change Points ◽  
Series Data ◽  
Attention Networks ◽  
Multi Stage ◽  
Detection Approach ◽  
Point Detection

Time series change point detection can identify the locations of abrupt points in many dynamic processes. It can help us to find anomaly data in an early stage. At the same time, detecting change points for long, periodic, and multiple input series data has received a lot of attention recently, and is universally applicable in many fields including power, environment, finance, and medicine. However, the performance of classical methods typically scales poorly for such time series. In this paper, we propose CPMAN, a novel prediction-based change point detection approach via multi-stage attention networks. Our model consists of two key modules. First, in the time series prediction module, we employ the multi-stage attention-based networks and integrate the multi-series fusion mechanism. This module can adaptively extract features from the relevant input series and capture the long-term temporal dependencies. Secondly, in the change point detection module, we use the wavelet analysis-based algorithm to detect change points efficiently and identify the change points and outliers. Extensive experiments are conducted on various real-world datasets and synthetic datasets, proving the superiority and effectiveness of CPMAN. Our approach outperforms the state-of-the-art methods by up to 12.1% on the F1 Score.

Sign in / Sign up

Export Citation Format

Share Document