scholarly journals Prediction of stock switching points by financial news

2021 ◽  
Author(s):  
Saeede Sadat Asadi Kakhki
Keyword(s):  
Change Point ◽  
Latent Dirichlet Allocation ◽  
Performance Metrics ◽  
Change Point Detection ◽  
Change Points ◽  
Detection Methods ◽  
Financial News ◽  
Point Detection ◽  
Stock Change ◽  
Random Guessing

The purpose of this study is to detect stock switching points from historical stock data and analyze corresponding financial news to predict upcoming stock switching points. Various change point detection methods have been investigated in the literature, such as online bayesian change point detection technique. Prediction of stock changing points using financial news has been implemented by different types of text mining techniques. In this study, online bayesian change point detection is implemented to detect stock switching points from historical stock data. Relevant news to detected change points are retrieved in the past and Latent Dirichlet Allocation technique is used to learn the hidden structures in the news data. Unseen news are then transferred to the trained topic representation. Similarity of relevant news and unseen news are used for prediction of future stock change points. Results show that stock switching points can be detected by historical stock data with better performance comparing to random guessing. It is possible to predict stock switching points by only fraction of financial news and with good result in terms of common performance metrics. According to this research, traders can take advantage of financial news to enhance prediction of future stock switching points.

scholarly journals Prediction of stock switching points by financial news

2021 ◽  
Author(s):  
Saeede Sadat Asadi Kakhki
Keyword(s):  
Change Point ◽  
Latent Dirichlet Allocation ◽  
Performance Metrics ◽  
Change Point Detection ◽  
Change Points ◽  
Detection Methods ◽  
Financial News ◽  
Point Detection ◽  
Stock Change ◽  
Random Guessing

The purpose of this study is to detect stock switching points from historical stock data and analyze corresponding financial news to predict upcoming stock switching points. Various change point detection methods have been investigated in the literature, such as online bayesian change point detection technique. Prediction of stock changing points using financial news has been implemented by different types of text mining techniques. In this study, online bayesian change point detection is implemented to detect stock switching points from historical stock data. Relevant news to detected change points are retrieved in the past and Latent Dirichlet Allocation technique is used to learn the hidden structures in the news data. Unseen news are then transferred to the trained topic representation. Similarity of relevant news and unseen news are used for prediction of future stock change points. Results show that stock switching points can be detected by historical stock data with better performance comparing to random guessing. It is possible to predict stock switching points by only fraction of financial news and with good result in terms of common performance metrics. According to this research, traders can take advantage of financial news to enhance prediction of future stock switching points.

scholarly journals Change-point detection in wind turbine SCADA data for robust condition monitoring with normal behaviour models

2020 ◽  
Author(s):  
Simon Letzgus
Keyword(s):  
Wind Turbine ◽  
Change Point ◽  
Significant Loss ◽  
Change Point Detection ◽  
Training Data ◽  
Change Points ◽  
Detection Methods ◽  
Full Potential ◽  
Normal Behaviour ◽  
Point Detection

Abstract. Analysis of data from wind turbine supervisory control and data acquisition (SCADA) systems has attracted considerable research interest in recent years. The data is predominantly used to gain insights into turbine condition without the need for additional sensing equipment. Most successful approaches apply semi-supervised anomaly detection methods, also called normal behaivour models, that use clean training data sets to establish healthy component baseline models. However, one of the major challenges when working with wind turbine SCADA data in practice is the presence of systematic changes in signal behaviour induced by malfunctions or maintenance actions. Even though this problem is well described in literature it has not been systematically addressed so far. This contribution is the first to comprehensively analyse the presence of change-points in wind turbine SCADA signals and introduce an algorithm for their automated detection. 600 signals from 33 turbines are analysed over an operational period of more than two years. During this time one third of the signals are affected by change-points. Kernel change-point detection methods have shown promising results in similar settings but their performance strongly depends on the choice of several hyperparameters. This contribution presents a comprehensive comparison between different kernels as well as kernel-bandwidth and regularisation-penalty selection heuristics. Moreover, an appropriate data pre-processing procedure is introduced. The results show that the combination of Laplace kernels with a newly introduced bandwidth and penalty selection heuristic robustly outperforms existing methods. In a signal validation setting more than 90 % of the signals were classified correctly regarding the presence or absence of change-points, resulting in a F1-score of 0.86. For a change-point-free sequence selection the most severe 60 % of all CPs could be automatically removed with a precision of more than 0.96 and therefore without a significant loss of training data. These results indicate that the algorithm can be a meaningful step towards automated SCADA data pre-processing which is key for data driven methods to reach their full potential. The algorithm is open source and its implementation in Python publicly available.

scholarly journals Change-point detection in wind turbine SCADA data for robust condition monitoring with normal behaviour models

2020 ◽  
Vol 5 (4) ◽  
pp. 1375-1397
Author(s):  
Simon Letzgus
Keyword(s):  
Wind Turbine ◽  
Change Point ◽  
Data Preprocessing ◽  
Change Point Detection ◽  
Training Data ◽  
Change Points ◽  
Detection Methods ◽  
Normal Behaviour ◽  
Behaviour Models ◽  
Point Detection

Abstract. Analysis of data from wind turbine supervisory control and data acquisition (SCADA) systems has attracted considerable research interest in recent years. Its predominant application is to monitor turbine condition without the need for additional sensing equipment. Most approaches apply semi-supervised anomaly detection methods, also called normal behaviour models, that require clean training data sets to establish healthy component baseline models. In practice, however, the presence of change points induced by malfunctions or maintenance actions poses a major challenge. Even though this problem is well described in literature, this contribution is the first to systematically evaluate and address the issue. A total of 600 signals from 33 turbines are analysed over an operational period of more than 2 years. During this time one-third of the signals were affected by change points, which highlights the necessity of an automated detection method. Kernel-based change-point detection methods have shown promising results in similar settings. We, therefore, introduce an appropriate SCADA data preprocessing procedure to ensure their feasibility and conduct comprehensive comparisons across several hyperparameter choices. The results show that the combination of Laplace kernels with a newly introduced bandwidth and regularisation-penalty selection heuristic robustly outperforms existing methods. More than 90 % of the signals were classified correctly regarding the presence or absence of change points, resulting in an F1 score of 0.86. For an automated change-point-free sequence selection, the most severe 60 % of all change points (CPs) could be automatically removed with a precision of more than 0.96 and therefore without any significant loss of training data. These results indicate that the algorithm can be a meaningful step towards automated SCADA data preprocessing, which is key for data-driven methods to reach their full potential. The algorithm is open source and its implementation in Python is publicly available.

scholarly journals Change Point Enhanced Anomaly Detection for IoT Time Series Data

Water ◽  
2021 ◽  
Vol 13 (12) ◽  
pp. 1633
Author(s):  
Elena-Simona Apostol ◽  
Ciprian-Octavian Truică ◽  
Florin Pop ◽  
Christian Esposito
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Change Point ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Change Point Detection ◽  
Change Points ◽  
Series Data ◽  
Prediction And Forecasting ◽  
Point Detection

Due to the exponential growth of the Internet of Things networks and the massive amount of time series data collected from these networks, it is essential to apply efficient methods for Big Data analysis in order to extract meaningful information and statistics. Anomaly detection is an important part of time series analysis, improving the quality of further analysis, such as prediction and forecasting. Thus, detecting sudden change points with normal behavior and using them to discriminate between abnormal behavior, i.e., outliers, is a crucial step used to minimize the false positive rate and to build accurate machine learning models for prediction and forecasting. In this paper, we propose a rule-based decision system that enhances anomaly detection in multivariate time series using change point detection. Our architecture uses a pipeline that automatically manages to detect real anomalies and remove the false positives introduced by change points. We employ both traditional and deep learning unsupervised algorithms, in total, five anomaly detection and five change point detection algorithms. Additionally, we propose a new confidence metric based on the support for a time series point to be an anomaly and the support for the same point to be a change point. In our experiments, we use a large real-world dataset containing multivariate time series about water consumption collected from smart meters. As an evaluation metric, we use Mean Absolute Error (MAE). The low MAE values show that the algorithms accurately determine anomalies and change points. The experimental results strengthen our assumption that anomaly detection can be improved by determining and removing change points as well as validates the correctness of our proposed rules in real-world scenarios. Furthermore, the proposed rule-based decision support systems enable users to make informed decisions regarding the status of the water distribution network and perform effectively predictive and proactive maintenance.

scholarly journals On the Performances of Trend and Change-Point Detection Methods for Remote Sensing Data

2020 ◽  
Vol 12 (6) ◽  
pp. 1008 ◽  
Author(s):  
Ana Militino ◽  
Mehdi Moradi ◽  
M. Ugarte
Keyword(s):  
Error Probability ◽  
Type I Error ◽  
Remote Sensing Data ◽  
Change Point Detection ◽  
Change Points ◽  
Detection Methods ◽  
Type I ◽  
Power Of The Test ◽  
Type I Error Probability ◽  
Point Detection

Detecting change-points and trends are common tasks in the analysis of remote sensing data. Over the years, many different methods have been proposed for those purposes, including (modified) Mann–Kendall and Cox–Stuart tests for detecting trends; and Pettitt, Buishand range, Buishand U, standard normal homogeneity (Snh), Meanvar, structure change (Strucchange), breaks for additive season and trend (BFAST), and hierarchical divisive (E.divisive) for detecting change-points. In this paper, we describe a simulation study based on including different artificial, abrupt changes at different time-periods of image time series to assess the performances of such methods. The power of the test, type I error probability, and mean absolute error (MAE) were used as performance criteria, although MAE was only calculated for change-point detection methods. The study reveals that if the magnitude of change (or trend slope) is high, and/or the change does not occur in the first or last time-periods, the methods generally have a high power and a low MAE. However, in the presence of temporal autocorrelation, MAE raises, and the probability of introducing false positives increases noticeably. The modified versions of the Mann–Kendall method for autocorrelated data reduce/moderate its type I error probability, but this reduction comes with an important power diminution. In conclusion, taking a trade-off between the power of the test and type I error probability, we conclude that the original Mann–Kendall test is generally the preferable choice. Although Mann–Kendall is not able to identify the time-period of abrupt changes, it is more reliable than other methods when detecting the existence of such changes. Finally, we look for trend/change-points in land surface temperature (LST), day and night, via monthly MODIS images in Navarre, Spain, from January 2001 to December 2018.

scholarly journals Locally adaptive change-point detection (LACPD) with applications to environmental changes

2021 ◽  
Author(s):  
Mehdi Moradi ◽  
Manuel Montesino-SanMartin ◽  
M. Dolores Ugarte ◽  
Ana F. Militino
Keyword(s):  
Time Series ◽  
Change Point ◽  
Error Probability ◽  
Type I Error ◽  
Change Point Detection ◽  
Alternative Methods ◽  
Change Points ◽  
Type I ◽  
Type I Error Probability ◽  
Point Detection

AbstractWe propose an adaptive-sliding-window approach (LACPD) for the problem of change-point detection in a set of time-ordered observations. The proposed method is combined with sub-sampling techniques to compensate for the lack of enough data near the time series’ tails. Through a simulation study, we analyse its behaviour in the presence of an early/middle/late change-point in the mean, and compare its performance with some of the frequently used and recently developed change-point detection methods in terms of power, type I error probability, area under the ROC curves (AUC), absolute bias, variance, and root-mean-square error (RMSE). We conclude that LACPD outperforms other methods by maintaining a low type I error probability. Unlike some other methods, the performance of LACPD does not depend on the time index of change-points, and it generally has lower bias than other alternative methods. Moreover, in terms of variance and RMSE, it outperforms other methods when change-points are close to the time series’ tails, whereas it shows a similar (sometimes slightly poorer) performance as other methods when change-points are close to the middle of time series. Finally, we apply our proposal to two sets of real data: the well-known example of annual flow of the Nile river in Awsan, Egypt, from 1871 to 1970, and a novel remote sensing data application consisting of a 34-year time-series of satellite images of the Normalised Difference Vegetation Index in Wadi As-Sirham valley, Saudi Arabia, from 1986 to 2019. We conclude that LACPD shows a good performance in detecting the presence of a change as well as the time and magnitude of change in real conditions.

scholarly journals A Remote Sensing-Based Assessment of Water Resources in the Arabian Peninsula

2021 ◽  
Vol 13 (2) ◽  
pp. 247
Author(s):  
Youssef Wehbe ◽  
Marouane Temimi
Keyword(s):  
Remote Sensing ◽  
Soil Moisture ◽  
Water Resources ◽  
Change Point ◽  
Arabian Peninsula ◽  
Change Point Detection ◽  
Change Points ◽  
Point Detection ◽  
Precipitation Trends

A better understanding of the spatiotemporal distribution of water resources is crucial for the sustainable development of hyper-arid regions. Here, we focus on the Arabian Peninsula (AP) and use remotely sensed data to (i) analyze the local climatology of total water storage (TWS), precipitation, and soil moisture; (ii) characterize their temporal variability and spatial distribution; and (iii) infer recent trends and change points within their time series. Remote sensing data for TWS, precipitation, and soil moisture are obtained from the Gravity Recovery and Climate Experiment (GRACE), the Tropical Rainfall Measuring Mission (TRMM), and the Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E), respectively. The study relies on trend analysis, the modified Mann–Kendall test, and change point detection statistics. We first derive 10-year (2002–2011) seasonal averages from each of the datasets and intercompare their spatial organization. In the absence of large-scale in situ data, we then compare trends from GRACE TWS retrievals to in situ groundwater observations locally over the subdomain of the United Arab Emirates (UAE). TWS anomalies vary between −6.2 to 3.2 cm/month and −6.8 to −0.3 cm/month during the winter and summer periods, respectively. Trend analysis shows decreasing precipitation trends (−2.3 × 10−4 mm/day) spatially aligned with decreasing soil moisture trends (−1.5 × 10−4 g/cm3/month) over the southern part of the AP, whereas the highest decreasing TWS trends (−8.6 × 10−2 cm/month) are recorded over areas of excessive groundwater extraction in the northern AP. Interestingly, change point detection reveals increasing precipitation trends pre- and post-change point breaks over the entire AP region. Significant spatial dependencies are observed between TRMM and GRACE change points, particularly over Yemen during 2010, revealing the dominant impact of climatic changes on TWS depletion.

Sign in / Sign up

Export Citation Format

Share Document