Optimization of SMOTE for imbalanced data based on AdaRBFNN and hybrid metaheuristics

2021 ◽  
Vol 25 (3) ◽  
pp. 541-554
Author(s):  
Zicheng Wang ◽  
Yanrui Sun
Keyword(s):  
Optimization Algorithm ◽  
Fitness Function ◽  
Imbalanced Data ◽  
Classification Performance ◽  
Rbf Neural Networks ◽  
Minority Class ◽  
Imbalanced Classification ◽  
Adaboost Algorithm ◽  
Candidate Solution

Oversampling ratio N and the minority class’ nearest neighboring number k are key hyperparameters of synthetic minority oversampling technique (SMOTE) to reconstruct the class distribution of dataset. No optimal default value exists there. Therefore, it is of necessity to discuss the influence of the output dataset on the classification performance when SMOTE adopts various hyperparameter combinations. In this paper, we propose a hyperparameter optimization algorithm for imbalanced data. By iterating to find reasonable N and k for SMOTE, so as to build a balanced and high-quality dataset. As a result, a model with outstanding performance and strong generalization ability is trained, thus effectively solving imbalanced classification. The proposed algorithm is based on the hybridization of simulated annealing mechanism (SA) and particle swarm optimization algorithm (PSO). In the optimization, Cohen’s Kappa is used to construct the fitness function, and AdaRBFNN, a new classifier, is integrated by multiple trained RBF neural networks based on AdaBoost algorithm. Kappa of each generation is calculated according to the classification results, so as to evaluate the quality of candidate solution. Experiments are conducted on seven groups of KEEL datasets. Results show that the proposed algorithm delivers excellent performance and can significantly improve the classification accuracy of the minority class.

A two-stage clustering-based cold-start method for active learning

2021 ◽  
Vol 25 (5) ◽  
pp. 1169-1185
Author(s):  
Deniu He ◽  
Hong Yu ◽  
Guoyin Wang ◽  
Jie Li
Keyword(s):  
Active Learning ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Cold Start ◽  
Classification Performance ◽  
The Novel ◽  
Two Stage ◽  
Minority Class ◽  
Novel Method ◽  
Multiple Clusters

The problem of initialization of active learning is considered in this paper. Especially, this paper studies the problem in an imbalanced data scenario, which is called as class-imbalance active learning cold-start. The novel method is two-stage clustering-based active learning cold-start (ALCS). In the first stage, to separate the instances of minority class from that of majority class, a multi-center clustering is constructed based on a new inter-cluster tightness measure, thus the data is grouped into multiple clusters. Then, in the second stage, the initial training instances are selected from each cluster based on an adaptive candidate representative instances determination mechanism and a clusters-cyclic instance query mechanism. The comprehensive experiments demonstrate the effectiveness of the proposed method from the aspects of class coverage, classification performance, and impact on active learning.

An Efficient Chaotic Salp Swarm Optimization Approach Based on Ensemble Algorithm for Solving Class Imbalance Problem

2021 ◽  
Author(s):  
Rekha G ◽  
Krishna Reddy V ◽  
chandrashekar jatoth ◽  
Ugo Fiore
Keyword(s):  
Feature Selection ◽  
Optimization Algorithm ◽  
Optimization Problems ◽  
Fitness Function ◽  
Class Imbalance ◽  
Optimization Approach ◽  
Class Imbalance Problem ◽  
Swarm Optimization ◽  
Adaboost Algorithm ◽  
Wide Range

Abstract Class imbalance problems have attracted the research community but a few works have focused on feature selection with imbalanced datasets. To handle class imbalance problems, we developed a novel fitness function for feature selection using the chaotic salp swarm optimization algorithm, an efficient meta-heuristic optimization algorithm that has been successfully used in a wide range of optimization problems. This paper proposes an Adaboost algorithm with chaotic salp swarm optimization. The most discriminating features are selected using salp swarm optimization and Adaboost classifiers are thereafter trained on the features selected. Experiments show the ability of the proposed technique to find the optimal features with performance maximization of Adaboost.

scholarly journals SMOTE-BD: An Exact and Scalable Oversampling Method for Imbalanced Classification in Big Data

2018 ◽  
Vol 18 (03) ◽  
pp. e23 ◽  
Author(s):  
María José Basgall ◽  
Waldo Hasperué ◽  
Marcelo Naiouf ◽  
Alberto Fernández ◽  
Francisco Herrera
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Model Design ◽  
Minority Class ◽  
Imbalanced Classification ◽  
Design And Implementation ◽  
Learning Issues ◽  
Intelligent Model

The volume of data in today's applications has meant a change in the way Machine Learning issues are addressed. Indeed, the Big Data scenario involves scalability constraints that can only be achieved through intelligent model design and the use of distributed technologies. In this context, solutions based on the Spark platform have established themselves as a de facto standard. In this contribution, we focus on a very important framework within Big Data Analytics, namely classification with imbalanced datasets. The main characteristic of this problem is that one of the classes is underrepresented, and therefore it is usually more complex to find a model that identifies it correctly. For this reason, it is common to apply preprocessing techniques such as oversampling to balance the distribution of examples in classes. In this work we present SMOTE-BD, a fully scalable preprocessing approach for imbalanced classification in Big Data. It is based on one of the most widespread preprocessing solutions for imbalanced classification, namely the SMOTE algorithm, which creates new synthetic instances according to the neighborhood of each example of the minority class. Our novel development is made to be independent of the number of partitions or processes created to achieve a higher degree of efficiency. Experiments conducted on different standard and Big Data datasets show the quality of the proposed design and implementation.

Evolutionary Undersampling for Classification with Imbalanced Datasets: Proposals and Taxonomy

2009 ◽  
Vol 17 (3) ◽  
pp. 275-306 ◽  
Author(s):  
Salvador García ◽  
Francisco Herrera
Keyword(s):  
Fitness Function ◽  
Imbalanced Data ◽  
Selection Procedure ◽  
Prototype Selection ◽  
Imbalanced Datasets ◽  
Classification Rate ◽  
Minority Class ◽  
Good Trade ◽  
And Performance ◽  
Nonparametric Statistical Procedures

Learning with imbalanced data is one of the recent challenges in machine learning. Various solutions have been proposed in order to find a treatment for this problem, such as modifying methods or the application of a preprocessing stage. Within the preprocessing focused on balancing data, two tendencies exist: reduce the set of examples (undersampling) or replicate minority class examples (oversampling). Undersampling with imbalanced datasets could be considered as a prototype selection procedure with the purpose of balancing datasets to achieve a high classification rate, avoiding the bias toward majority class examples. Evolutionary algorithms have been used for classical prototype selection showing good results, where the fitness function is associated to the classification and reduction rates. In this paper, we propose a set of methods called evolutionary undersampling that take into consideration the nature of the problem and use different fitness functions for getting a good trade-off between balance of distribution of classes and performance. The study includes a taxonomy of the approaches and an overall comparison among our models and state of the art undersampling methods. The results have been contrasted by using nonparametric statistical procedures and show that evolutionary undersampling outperforms the nonevolutionary models when the degree of imbalance is increased.

scholarly journals Imbalanced Data Sets Classification Based on SVM for Sand-Dust Storm Warning

2015 ◽  
Vol 2015 ◽  
pp. 1-8
Author(s):  
Yonghua Xie ◽  
Yurong Liu ◽  
Qingqiu Fu
Keyword(s):  
Dust Storm ◽  
Adaptive Sampling ◽  
Imbalanced Data ◽  
Real Data ◽  
Classification Performance ◽  
Selection Strategy ◽  
Data Sets ◽  
Minority Class ◽  
Redundant Data ◽  
Sand Dust

In view of the SVM classification for the imbalanced sand-dust storm data sets, this paper proposes a hybrid self-adaptive sampling method named SRU-AIBSMOTE algorithm. This method can adaptively adjust neighboring selection strategy based on the internal distribution of sample sets. It produces virtual minority class instances through randomized interpolation in the spherical space which consists of minority class instances and their neighbors. The random undersampling is also applied to undersample the majority class instances for removal of redundant data in the sample sets. The comparative experimental results on the real data sets from Yanchi and Tongxin districts in Ningxia of China show that the SRU-AIBSMOTE method can obtain better classification performance than some traditional classification methods.

scholarly journals Granular Classification for Imbalanced Datasets: A Minkowski Distance-Based Method

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 54
Author(s):  
Chen Fu ◽  
Jianhua Yang
Keyword(s):  
Imbalanced Data ◽  
Main Idea ◽  
Fuzzy Rule ◽  
Classification Performance ◽  
Distance Measures ◽  
Minkowski Distance ◽  
Imbalanced Datasets ◽  
Minority Class ◽  
Information Granules ◽  
Practical Applications

The problem of classification for imbalanced datasets is frequently encountered in practical applications. The data to be classified in this problem are skewed, i.e., the samples of one class (the minority class) are much less than those of other classes (the majority class). When dealing with imbalanced datasets, most classifiers encounter a common limitation, that is, they often obtain better classification performances on the majority classes than those on the minority class. To alleviate the limitation, in this study, a fuzzy rule-based modeling approach using information granules is proposed. Information granules, as some entities derived and abstracted from data, can be used to describe and capture the characteristics (distribution and structure) of data from both majority and minority classes. Since the geometric characteristics of information granules depend on the distance measures used in the granulation process, the main idea of this study is to construct information granules on each class of imbalanced data using Minkowski distance measures and then to establish the classification models by using “If-Then” rules. The experimental results involving synthetic and publicly available datasets reflect that the proposed Minkowski distance-based method can produce information granules with a series of geometric shapes and construct granular models with satisfying classification performance for imbalanced datasets.

scholarly journals Comprehensive Assessment of Imbalanced Data Classification

2020 ◽  
Vol 9 (4) ◽  
pp. 1426-1431
Keyword(s):  
Real World ◽  
Imbalanced Data ◽  
Predictive Modelling ◽  
Classification Problem ◽  
Minority Class ◽  
Unequal Distribution ◽  
Imbalanced Classification ◽  
Imbalanced Data Classification ◽  
Improved Performance ◽  
And Performance

This is an attempt to address the various challenges opportunities and scope for formulating and designing new procedure in imbalanced classification problem which poses a challenge to a predictive modelling as many of AI ML n DL algorithms which are extensively used for classification are always designed from the perspective of with majority of focus on assuming equal number of examples for a class. It leads to poor efficiency and performance especially in minority class. As Minority class is always very crucial and sensitive to classification errors and also its utmost important in imbalanced classification. This chapter discusses addresses and gives novel as well as deep insights with unequal distribution of classes in training datasets. Largely real time and real world classifications are comprising imbalanced distribution so need specialized techniques for more challenging and sophisticated models with minimal errors and improved performance.

scholarly journals Imbalanced Data Fault Diagnosis Based on an Evolutionary Online Sequential Extreme Learning Machine

Symmetry ◽  
2020 ◽  
Vol 12 (8) ◽  
pp. 1204 ◽  
Author(s):  
Wei Hao ◽  
Feng Liu
Keyword(s):  
Fault Diagnosis ◽  
Extreme Learning Machine ◽  
High Speed ◽  
Imbalanced Data ◽  
Classification Performance ◽  
Complex Data ◽  
Minority Class ◽  
Diagnosis Model ◽  
Learning Machine ◽  
Hidden Layer

To quickly and effectively identify an axle box bearing fault of high-speed electric multiple units (EMUs), an evolutionary online sequential extreme learning machine (OS-ELM) fault diagnosis method for imbalanced data was proposed. In this scheme, the resampling scale is first determined according to the resampling empirical formulation, the K-means synthetic minority oversampling technique (SMOTE) method is then used for oversampling the minority class samples, a method based on Euclidean distance is applied for undersampling the majority class samples, and the complex data features are extracted from the reconstructed dataset. Second, the reconstructed dataset is input into the diagnosis model. Finally, the artificial bee colony (ABC) algorithm is used to globally optimize the combination of input weights, hidden layer bias, and the number of hidden layer nodes for an OS-ELM, and the diagnosis model is allowed to evolve. The proposed method was tested on the axle box bearing monitoring data of high-speed EMUs, on which the position of the axle box bearings was symmetrical. Numerical testing proved that the method has the characteristics of faster detection and higher classification performance regarding the minority class data compared to other standard and classical algorithms.

scholarly journals A Novel Ensemble Framework Based on K-Means and Resampling for Imbalanced Data

2020 ◽  
Vol 10 (5) ◽  
pp. 1684
Author(s):  
Huajuan Duan ◽  
Yongqing Wei ◽  
Peiyu Liu ◽  
Hongxia Yin
Keyword(s):  
Evaluation Criteria ◽  
Imbalanced Data ◽  
Data Preprocessing ◽  
Ensemble Classifier ◽  
Data Sets ◽  
Imbalanced Datasets ◽  
Minority Class ◽  
K Value ◽  
Imbalanced Classification ◽  
The Past

Imbalanced classification is one of the most important problems of machine learning and data mining, existing in many real datasets. In the past, many basic classifiers such as SVM, KNN, and so on have been used for imbalanced datasets in which the number of one sample is larger than that of another, but the classification effect is not ideal. Some data preprocessing methods have been proposed to reduce the imbalance ratio of data sets and combine with the basic classifiers to get better performance. In order to improve the whole classification accuracy, we propose a novel classifier ensemble framework based on K-means and resampling technique (EKR). First, we divide the data samples in the majority class into several sub-clusters using K-means, k-value is determined by Average Silhouette Coefficient, and then adjust the number of data samples of each sub-cluster to be the same as that of the minority classes through resampling technology, after that each adjusted sub-cluster and the minority class are combined into several balanced subsets, the base classifier is trained on each balanced subset separately, and finally integrated into a strong ensemble classifier. In this paper, the extensive experimental results on 16 imbalanced datasets demonstrate the effectiveness and feasibility of the proposed algorithm in terms of multiple evaluation criteria, and EKR can achieve better performance when compared with several classical imbalanced classification algorithms using different data preprocessing methods.

scholarly journals Raking and Relabeling for Imbalanced Data

2022 ◽  
Author(s):  
Seunghwan Park ◽  
Hae-Wwan Lee ◽  
Jongho Im
Keyword(s):  
High Dimensional Data ◽  
Imbalanced Data ◽  
Sampling Strategy ◽  
Classification Performance ◽  
Mixed Data ◽  
Categorical Variables ◽  
High Dimensional ◽  
Data Generation ◽  
Minority Class ◽  
Imbalanced Data Classification

<div>We consider the binary classification of imbalanced data. A dataset is imbalanced if the proportion of classes are heavily skewed. Imbalanced data classification is often challengeable, especially for high-dimensional data, because unequal classes deteriorate classifier performance. Under sampling the majority class or oversampling the minority class are popular methods to construct balanced samples, facilitating classification performance improvement. However, many existing sampling methods cannot be easily extended to high-dimensional data and mixed data, including categorical variables, because they often require approximating the attribute distributions, which becomes another critical issue. In this paper, we propose a new sampling strategy employing raking and relabeling procedures, such that the attribute values of the majority class are imputed for the values of the minority class in the construction of balanced samples. The proposed algorithms produce comparable performance as existing popular methods but are more flexible regarding the data shape and attribute size. The sampling algorithm is attractive in practice, considering that it does not require density estimation for synthetic data generation in oversampling and is not bothered by mixed-type variables. In addition, the proposed sampling strategy is robust to classifiers in the sense that classification performance is not sensitive to choosing the classifiers.</div>

Sign in / Sign up

Export Citation Format

Share Document