scholarly journals MultiETSC: automated machine learning for early time series classification

2021 ◽  
Author(s):  
Gilles Ottervanger ◽  
Mitra Baratchi ◽  
Holger H. Hoos
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Optimal Algorithm ◽  
Empirical Evaluation ◽  
Early Time ◽  
Time Series Classification ◽  
Algorithm Selection ◽  
Trade Off ◽  
Conflicting Objectives ◽  
Automated Machine Learning

AbstractEarly time series classification (EarlyTSC) involves the prediction of a class label based on partial observation of a given time series. Most EarlyTSC algorithms consider the trade-off between accuracy and earliness as two competing objectives, using a single dedicated hyperparameter. To obtain insights into this trade-off requires finding a set of non-dominated (Pareto efficient) classifiers. So far, this has been approached through manual hyperparameter tuning. Since the trade-off hyperparameters only provide indirect control over the earliness-accuracy trade-off, manual tuning is tedious and tends to result in many sub-optimal hyperparameter settings. This complicates the search for optimal hyperparameter settings and forms a hurdle for the application of EarlyTSC to real-world problems. To address these issues, we propose an automated approach to hyperparameter tuning and algorithm selection for EarlyTSC, building on developments in the fast-moving research area known as automated machine learning (AutoML). To deal with the challenging task of optimising two conflicting objectives in early time series classification, we propose MultiETSC, a system for multi-objective algorithm selection and hyperparameter optimisation (MO-CASH) for EarlyTSC. MultiETSC can potentially leverage any existing or future EarlyTSC algorithm and produces a set of Pareto optimal algorithm configurations from which a user can choose a posteriori. As an additional benefit, our proposed framework can incorporate and leverage time-series classification algorithms not originally designed for EarlyTSC for improving performance on EarlyTSC; we demonstrate this property using a newly defined, “naïve” fixed-time algorithm. In an extensive empirical evaluation of our new approach on a benchmark of 115 data sets, we show that MultiETSC performs substantially better than baseline methods, ranking highest (avg. rank 1.98) compared to conceptually simpler single-algorithm (2.98) and single-objective alternatives (4.36).

scholarly journals Development of Indicator of Data Sufficiency for Feature-based Early Time Series Classification with Applications of Bearing Fault Diagnosis

Processes ◽  
2020 ◽  
Vol 8 (7) ◽  
pp. 790
Author(s):  
Gilseung Ahn ◽  
Hwanchul Lee ◽  
Jisu Park ◽  
Sun Hur
Keyword(s):  
Time Series ◽  
Fault Diagnosis ◽  
Early Time ◽  
Time Series Classification ◽  
Bearing Faults ◽  
Bearing Fault ◽  
Bearing Fault Diagnosis ◽  
Benchmark Datasets ◽  
Feature Based ◽  
General Time

Diagnosis of bearing faults is crucial in various industries. Time series classification (TSC) assigns each time series to one of a set of pre-defined classes, such as normal and fault, and has been regarded as an appropriate approach for bearing fault diagnosis. Considering late and inaccurate fault diagnosis may have a significant impact on maintenance costs, it is important to classify bearing signals as early and accurately as possible. TSC, however, has a major limitation, which is that a time series cannot be classified until the entire series is collected, implying that a fault cannot be diagnosed using TSC in advance. Therefore, it is important to classify a partially collected time series for early time series classification (ESTC), which is a TSC that considers both accuracy and earliness. Feature-based TSCs can handle this, but the problem is to determine whether a partially collected time series is enough for a decision that is still unsolved. Motivated by this, we propose an indicator of data sufficiency to determine whether a feature-based fault detection classifier can start classifying partially collected signals in order to diagnose bearing faults as early and accurately as possible. The indicator is trained based on the cosine similarity between signals that were collected fully and partially as input to the classifier. In addition, a parameter setting method for efficiently training the indicator is also proposed. The results of experiments using four benchmark datasets verified that the proposed indicator increased both accuracy and earliness compared with the previous time series classification method and general time series classification.

scholarly journals When algorithm selection meets Bi-linear Learning to Rank: accuracy and inference time trade off with candidates expansion

2020 ◽  
Author(s):  
Jing Yuan ◽  
Christian Geissler ◽  
Weijia Shao ◽  
Andreas Lommatzsch ◽  
Brijnesh Jain
Keyword(s):  
Optimal Algorithm ◽  
Learning To Rank ◽  
Mapping Function ◽  
Evaluation Process ◽  
Objective Evaluation ◽  
Problem Instance ◽  
Cost Calculation ◽  
Algorithm Selection ◽  
Trade Off ◽  
Problem Instances

Abstract Algorithm selection (AS) tasks are dedicated to find the optimal algorithm for an unseen problem instance. With the knowledge of problem instances’ meta-features and algorithms’ landmark performances, Machine Learning (ML) approaches are applied to solve AS problems. However, the standard training process of benchmark ML approaches in AS either needs to train the models specifically for every algorithm or relies on the sparse one-hot encoding as the algorithms’ representation. To escape these intermediate steps and form the mapping function directly, we borrow the learning to rank framework from Recommender System (RS) and embed the bi-linear factorization to model the algorithms’ performances in AS. This Bi-linear Learning to Rank (BLR) has proven to work with competence in some AS scenarios and thus is also proposed as a benchmark approach. Thinking from the evaluation perspective in the modern AS challenges, precisely predicting the performance is usually the measuring goal. Though approaches’ inference time also needs to be counted for the running time cost calculation, it’s always overlooked in the evaluation process. The multi-objective evaluation metric Adjusted Ratio of Root Ratios (A3R) is therefore advocated in this paper to balance the trade-off between the accuracy and inference time in AS. Concerning A3R, BLR outperforms other benchmarks when expanding the candidates range to TOP3. The better effect of this candidates expansion results from the cumulative optimum performance during the AS process. We take the further step in the experimentation to represent the advantage of such TOPK expansion, and illustrate that such expansion can be considered as the supplement for the convention of TOP1 selection during the evaluation process.

scholarly journals DTEC: Distance Transformation Based Early Time Series Classification

2019 ◽  
pp. 486-494 ◽  
Author(s):  
Liuyi Yao ◽  
Yaliang Li ◽  
Yezheng Li ◽  
Hengtong Zhang ◽  
Mengdi Huai ◽  
...  
Keyword(s):  
Time Series ◽  
Early Time ◽  
Time Series Classification ◽  
Distance Transformation

scholarly journals TEASER: early and accurate time series classification

2020 ◽  
Vol 34 (5) ◽  
pp. 1336-1362
Author(s):  
Patrick Schäfer ◽  
Ulf Leser
Keyword(s):  
Time Series ◽  
Real Life ◽  
Early Time ◽  
Classification Problem ◽  
Critical Issue ◽  
Decision Time ◽  
Superior Performance ◽  
Optimal Decision ◽  
Time Series Classification ◽  
Start Time

Abstract Early time series classification (eTSC) is the problem of classifying a time series after as few measurements as possible with the highest possible accuracy. The most critical issue of any eTSC method is to decide when enough data of a time series has been seen to take a decision: Waiting for more data points usually makes the classification problem easier but delays the time in which a classification is made; in contrast, earlier classification has to cope with less input data, often leading to inferior accuracy. The state-of-the-art eTSC methods compute a fixed optimal decision time assuming that every times series has the same defined start time (like turning on a machine). However, in many real-life applications measurements start at arbitrary times (like measuring heartbeats of a patient), implying that the best time for taking a decision varies widely between time series. We present TEASER, a novel algorithm that models eTSC as a two-tier classification problem: In the first tier, a classifier periodically assesses the incoming time series to compute class probabilities. However, these class probabilities are only used as output label if a second-tier classifier decides that the predicted label is reliable enough, which can happen after a different number of measurements. In an evaluation using 45 benchmark datasets, TEASER is two to three times earlier at predictions than its competitors while reaching the same or an even higher classification accuracy. We further show TEASER’s superior performance using real-life use cases, namely energy monitoring, and gait detection.

scholarly journals Active Learning Framework for Time-Series Classification of Vibration and Industrial Process Data

2021 ◽  
Vol 13 (1) ◽  
Author(s):  
Sergio Martin del Campo Barraza ◽  
William Lindskog ◽  
Davide Badalotti ◽  
Oskar Liew ◽  
Arash Toyser
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Active Learning ◽  
Condition Monitoring ◽  
Industrial Process ◽  
Time Series Classification ◽  
Process Data ◽  
Learning Framework ◽  
Monitoring Applications

Data-based models built using machine learning solutions are becoming more prominent in the condition monitoring, maintenance, and prognostics fields. The capacity to build these models using a machine learning approach depends largely in the quality of the data. Of particular importance is the availability of labelled data, which describes the conditions that are intended to be identified. However, properly labelled data that is useful in many machine learning strategies is a scare resource. Furthermore, producing high-quality labelled data is expensive, time-consuming and a lot of times inaccurate given the uncertainty surrounding the labeling process and the annotators.  Active Learning (AL) has emerged as a semi-supervised approach that enables cost and time reductions of the labeling process. This approach has had a delayed adoption for time series classification given the difficulty to extract and present the time series information in such a way that it is easy to understand for the human annotator who incorporates the labels. This difficulty arises from the large dimensionality that many of these time series possess. This challenge is exacerbated by the cold-start problem, where the initial labelled dataset used in typical AL frameworks may not exist. Thus, the initial set of labels to be allocated to the time series samples is not available. This last challenge is particularly common on many condition monitoring applications where data samples of specific faults or problems does not exist. In this article, we present an AL framework to be used in the classification of time series from industrial process data, in particular vibration waveforms originated from condition monitoring applications. In this framework, we deal with the absence of labels to train an initial classification model by introducing a pre-clustering step. This step uses an unsupervised clustering algorithm to identify the number of labels and selects the points with a stronger group belonging as initial samples to be labelled in the active learning step. Furthermore, this framework presents two approaches to present the information to the annotator that can be via time-series imaging and automatic extraction of statistical features. Our work is motivated by the interest to facilitate the effort required for labeling time-series waveforms, while maintaining a high level of accuracy and consistency on those labels. In addition, we study the number of time-series samples that require to be labelled to achieve different levels of classification accuracy, as well as their confidence intervals. These experiments are carried out using vibration signals from a well-known rolling element bearing dataset and typical process data from a production plant.   An active learning framework that considers the conditions of the data commonly found in maintenance and condition monitoring applications while presenting the data in ways easy to interpret by human annotators can facilitate the generation reliable datasets. These datasets can, in turn, assist in the development of data-driven models that describe the many different processes that a machine undergoes.

Sign in / Sign up

Export Citation Format

Share Document