A two-stage clustering-based cold-start method for active learning

2021 ◽  
Vol 25 (5) ◽  
pp. 1169-1185
Author(s):  
Deniu He ◽  
Hong Yu ◽  
Guoyin Wang ◽  
Jie Li
Keyword(s):  
Active Learning ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Cold Start ◽  
Classification Performance ◽  
The Novel ◽  
Two Stage ◽  
Minority Class ◽  
Novel Method ◽  
Multiple Clusters

The problem of initialization of active learning is considered in this paper. Especially, this paper studies the problem in an imbalanced data scenario, which is called as class-imbalance active learning cold-start. The novel method is two-stage clustering-based active learning cold-start (ALCS). In the first stage, to separate the instances of minority class from that of majority class, a multi-center clustering is constructed based on a new inter-cluster tightness measure, thus the data is grouped into multiple clusters. Then, in the second stage, the initial training instances are selected from each cluster based on an adaptive candidate representative instances determination mechanism and a clusters-cyclic instance query mechanism. The comprehensive experiments demonstrate the effectiveness of the proposed method from the aspects of class coverage, classification performance, and impact on active learning.

scholarly journals Difficulty Factors and Preprocessing in Imbalanced Data Sets: An Experimental Study on Artificial Data

2017 ◽  
Vol 42 (2) ◽  
pp. 149-176 ◽  
Author(s):  
Szymon Wojciechowski ◽  
Szymon Wilk
Keyword(s):  
Experimental Study ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Classification Performance ◽  
Data Sets ◽  
Artificial Data ◽  
Minority Class ◽  
Imbalanced Data Sets ◽  
The Impact

Abstract In this paper we describe results of an experimental study where we checked the impact of various difficulty factors in imbalanced data sets on the performance of selected classifiers applied alone or combined with several preprocessing methods. In the study we used artificial data sets in order to systematically check factors such as dimensionality, class imbalance ratio or distribution of specific types of examples (safe, borderline, rare and outliers) in the minority class. The results revealed that the latter factor was the most critical one and it exacerbated other factors (in particular class imbalance). The best classification performance was demonstrated by non-symbolic classifiers, particular by k-NN classifiers (with 1 or 3 neighbors - 1NN and 3NN, respectively) and by SVM. Moreover, they benefited from different preprocessing methods - SVM and 1NN worked best with undersampling, while oversampling was more beneficial for 3NN.

scholarly journals Embedding Undersampling Rotation Forest for Imbalanced Problem

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Huaping Guo ◽  
Xiaoyu Diao ◽  
Hongbing Liu
Keyword(s):  
Imbalanced Data ◽  
Feature Space ◽  
Original Data ◽  
Training Set ◽  
Data Set ◽  
Minority Class ◽  
Rotation Forest ◽  
Novel Method ◽  
Individual Classifier ◽  
The Cost

Rotation Forest is an ensemble learning approach achieving better performance comparing to Bagging and Boosting through building accurate and diverse classifiers using rotated feature space. However, like other conventional classifiers, Rotation Forest does not work well on the imbalanced data which are characterized as having much less examples of one class (minority class) than the other (majority class), and the cost of misclassifying minority class examples is often much more expensive than the contrary cases. This paper proposes a novel method called Embedding Undersampling Rotation Forest (EURF) to handle this problem (1) sampling subsets from the majority class and learning a projection matrix from each subset and (2) obtaining training sets by projecting re-undersampling subsets of the original data set to new spaces defined by the matrices and constructing an individual classifier from each training set. For the first method, undersampling is to force the rotation matrix to better capture the features of the minority class without harming the diversity between individual classifiers. With respect to the second method, the undersampling technique aims to improve the performance of individual classifiers on the minority class. The experimental results show that EURF achieves significantly better performance comparing to other state-of-the-art methods.

scholarly journals Improving undersampling-based ensemble with rotation forest for imbalanced problem

2019 ◽  
pp. 1371-1386 ◽  
Author(s):  
Huaping GUO ◽  
Xiaoyu DIAO ◽  
Hongbing LIU
Keyword(s):  
High Performance ◽  
State Of The Art ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Ensemble Methods ◽  
Sampling Technique ◽  
Robust Methods ◽  
Limited Data ◽  
Minority Class ◽  
Rotation Forest

As one of the most challenging and attractive issues in pattern recognition and machine learning, the imbalanced problem has attracted increasing attention. For two-class data, imbalanced data are characterized by the size of one class (majority class) being much larger than that of the other class (minority class), which makes the constructed models focus more on the majority class and ignore or even misclassify the examples of the minority class. The undersampling-based ensemble, which learns individual classifiers from undersampled balanced data, is an effective method to cope with the class-imbalance data. The problem in this method is that the size of the dataset to train each classifier is notably small; thus, how to generate individual classifiers with high performance from the limited data is a key to the success of the method. In this paper, rotation forest (an ensemble method) is used to improve the performance of the undersampling-based ensemble on the imbalanced problem because rotation forest has higher performance than other ensemble methods such as bagging, boosting, and random forest, particularly for small-sized data. In addition, rotation forest is more sensitive to the sampling technique than some robust methods including SVM and neural networks; thus, it is easier to create individual classifiers with diversity using rotation forest. Two versions of the improved undersampling-based ensemble methods are implemented: 1) undersampling subsets from the majority class and learning each classifier using the rotation forest on the data obtained by combing each subset with the minority class and 2) similarly to the first method, with the exception of removing the majority class examples that are correctly classified with high confidence after learning each classifier for further consideration. The experimental results show that the proposed methods show significantly better performance on measures of recall, g-mean, f-measure, and AUC than other state-of-the-art methods on 30 datasets with various data distributions and different imbalance ratios.

scholarly journals Imbalanced Data Sets Classification Based on SVM for Sand-Dust Storm Warning

2015 ◽  
Vol 2015 ◽  
pp. 1-8
Author(s):  
Yonghua Xie ◽  
Yurong Liu ◽  
Qingqiu Fu
Keyword(s):  
Dust Storm ◽  
Adaptive Sampling ◽  
Imbalanced Data ◽  
Real Data ◽  
Classification Performance ◽  
Selection Strategy ◽  
Data Sets ◽  
Minority Class ◽  
Redundant Data ◽  
Sand Dust

In view of the SVM classification for the imbalanced sand-dust storm data sets, this paper proposes a hybrid self-adaptive sampling method named SRU-AIBSMOTE algorithm. This method can adaptively adjust neighboring selection strategy based on the internal distribution of sample sets. It produces virtual minority class instances through randomized interpolation in the spherical space which consists of minority class instances and their neighbors. The random undersampling is also applied to undersample the majority class instances for removal of redundant data in the sample sets. The comparative experimental results on the real data sets from Yanchi and Tongxin districts in Ningxia of China show that the SRU-AIBSMOTE method can obtain better classification performance than some traditional classification methods.

scholarly journals Granular Classification for Imbalanced Datasets: A Minkowski Distance-Based Method

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 54
Author(s):  
Chen Fu ◽  
Jianhua Yang
Keyword(s):  
Imbalanced Data ◽  
Main Idea ◽  
Fuzzy Rule ◽  
Classification Performance ◽  
Distance Measures ◽  
Minkowski Distance ◽  
Imbalanced Datasets ◽  
Minority Class ◽  
Information Granules ◽  
Practical Applications

The problem of classification for imbalanced datasets is frequently encountered in practical applications. The data to be classified in this problem are skewed, i.e., the samples of one class (the minority class) are much less than those of other classes (the majority class). When dealing with imbalanced datasets, most classifiers encounter a common limitation, that is, they often obtain better classification performances on the majority classes than those on the minority class. To alleviate the limitation, in this study, a fuzzy rule-based modeling approach using information granules is proposed. Information granules, as some entities derived and abstracted from data, can be used to describe and capture the characteristics (distribution and structure) of data from both majority and minority classes. Since the geometric characteristics of information granules depend on the distance measures used in the granulation process, the main idea of this study is to construct information granules on each class of imbalanced data using Minkowski distance measures and then to establish the classification models by using “If-Then” rules. The experimental results involving synthetic and publicly available datasets reflect that the proposed Minkowski distance-based method can produce information granules with a series of geometric shapes and construct granular models with satisfying classification performance for imbalanced datasets.

scholarly journals DATA IMBALANCE IN LANDSLIDE SUSCEPTIBILITY ZONATION: UNDER-SAMPLING FOR CLASS-IMBALANCE LEARNING

2020 ◽  
Vol XLII-3/W11 ◽  
pp. 51-57
Author(s):  
S. K. Gupta ◽  
M. Jhunjhunwalla ◽  
A. Bhardwaj ◽  
D. P. Shukla
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Training Data ◽  
Support Vector ◽  
Fisher Discriminant Analysis ◽  
Minority Class ◽  
Data Imbalance ◽  
Artificial Neural

Abstract. Machine learning methods such as artificial neural network, support vector machine etc. require a large amount of training data, however, the number of landslide occurrences are limited in a study area. The limited number of landslides leads to a small number of positive class pixels in the training data. On contrary, the number of non-landslide pixels (negative class pixels) are enormous in numbers. This under-represented data and severe class distribution skew create a data imbalance for learning algorithms and suboptimal models, which are biased towards the majority class (non-landslide pixels) and have low performance on the minority class (landslide pixels).In this work, we have used two algorithms namely EasyEnsemble and BalanceCascade for balancing the data. This balanced data is used with feature selection methods such as fisher discriminant analysis (FDA), logistic regression (LR) and artificial neural network (ANN) to generate LSZ maps The results of the study show that ANN with balanced data has major improvements in preparation of susceptibility maps over imbalanced data, where as the LR method is ill-effected by data balancing algorithms. The FDA does not show significant changes between balanced and imbalanced data.

A Multi-Objective Ensemble Method for Class Imbalance Learning

2017 ◽  
Vol 2 (1) ◽  
pp. 16-34
Author(s):  
Sajad Emamipour ◽  
Rasoul Sali ◽  
Zahra Yousefi
Keyword(s):  
Class Imbalance ◽  
Imbalanced Data ◽  
Classification Performance ◽  
Ensemble Classifiers ◽  
Feature Selection Technique ◽  
Multi Objective ◽  
Proposed Model ◽  
Training Examples ◽  
Imbalance Learning ◽  
Class Imbalance Learning

This article describes how class imbalance learning has attracted great attention in recent years as many real world domain applications suffer from this problem. Imbalanced class distribution occurs when the number of training examples for one class far surpasses the training examples of the other class often the one that is of more interest. This problem may produce an important deterioration of the classifier performance, in particular with patterns belonging to the less represented classes. Toward this end, the authors developed a hybrid model to address the class imbalance learning with focus on binary class problems. This model combines benefits of the ensemble classifiers with a multi objective feature selection technique to achieve higher classification performance. The authors' model also proposes non-dominated sets of features. Then they evaluate the performance of the proposed model by comparing its results with notable algorithms for solving imbalanced data problem. Finally, the authors utilize the proposed model in medical domain of predicting life expectancy in post-operative of thoracic surgery patients.

Class imbalance in gradient boosting classification algorithms: Application to experimental stroke data

2020 ◽  
pp. 096228022098048
Author(s):  
Olga Lyashevska ◽  
Fiona Malone ◽  
Eugene MacCarthy ◽  
Jens Fiehler ◽  
Jan-Hendrik Buhk ◽  
...  
Keyword(s):  
Class Imbalance ◽  
Imbalanced Data ◽  
Medical Data ◽  
Interactive Effects ◽  
Experimental Stroke ◽  
Gradient Boosting ◽  
Classification Algorithms ◽  
Minority Class ◽  
Linear Relationships ◽  
Boosting Algorithm

Imbalance between positive and negative outcomes, a so-called class imbalance, is a problem generally found in medical data. Imbalanced data hinder the performance of conventional classification methods which aim to improve the overall accuracy of the model without accounting for uneven distribution of the classes. To rectify this, the data can be resampled by oversampling the positive (minority) class until the classes are approximately equally represented. After that, a prediction model such as gradient boosting algorithm can be fitted with greater confidence. This classification method allows for non-linear relationships and deep interactive effects while focusing on difficult areas by iterative shifting towards problematic observations. In this study, we demonstrate application of these methods to medical data and develop a practical framework for evaluation of features contributing into the probability of stroke.

scholarly journals VFC-SMOTE: very fast continuous synthetic minority oversampling for evolving data streams

2021 ◽  
Author(s):  
Alessio Bernardo ◽  
Emanuele Della Valle
Keyword(s):  
Data Streams ◽  
Concept Drift ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Real Data ◽  
Minority Class ◽  
Machine Learning Classification ◽  
Imbalance Learning ◽  
Class Imbalance Learning ◽  
Better Than

AbstractThe world is constantly changing, and so are the massive amount of data produced. However, only a few studies deal with online class imbalance learning that combines the challenges of class-imbalanced data streams and concept drift. In this paper, we propose the very fast continuous synthetic minority oversampling technique (VFC-SMOTE). It is a novel meta-strategy to be prepended to any streaming machine learning classification algorithm aiming at oversampling the minority class using a new version of Smote and Borderline-Smote inspired by Data Sketching. We benchmarked VFC-SMOTE pipelines on synthetic and real data streams containing different concept drifts, imbalance levels, and class distributions. We bring statistical evidence that VFC-SMOTE pipelines learn models whose minority class performances are better than state-of-the-art. Moreover, we analyze the time/memory consumption and the concept drift recovery speed.

scholarly journals Oversampling Imbalanced Data Based on Convergent WGAN for Network Threat Detection

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Yanping Xu ◽  
Xiaoyu Zhang ◽  
Zhenliang Qiu ◽  
Xia Zhang ◽  
Jian Qiu ◽  
...  
Keyword(s):  
Nash Equilibrium ◽  
Loss Function ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Real Data ◽  
Threat Detection ◽  
Training Process ◽  
Generative Adversarial Network ◽  
Minority Class ◽  
Adversarial Network

Class imbalance is a common problem in network threat detection. Oversampling the minority class is regarded as a popular countermeasure by generating enough new minority samples. Generative adversarial network (GAN) is a typical generative model that can generate any number of artificial minority samples, which are close to the real data. However, it is difficult to train GAN, and the Nash equilibrium is almost impossible to achieve. Therefore, in order to improve the training stability of GAN for oversampling to detect the network threat, a convergent WGAN-based oversampling model called convergent WGAN (CWGAN) is proposed in this paper. The training process of CWGAN contains multiple iterations. In each iteration, the training epochs of the discriminator are dynamic, which is determined by the convergence of discriminator loss function in the last two iterations. When the discriminator is trained to convergence, the generator will then be trained to generate new minority samples. The experiment results show that CWGAN not only improve the training stability of WGAN on the loss smoother and closer to 0 but also improve the performance of the minority class through oversampling, which means that CWGAN can improve the performance of network threat detection.

Sign in / Sign up

Export Citation Format

Share Document